Senior Manager - Data Engineering

Mercer

7 - 11 years

13 - 18 Lacs

pune

Posted:2 weeks ago| Platform:

Apply

Skills Required

cloud integration amazon sqs lambda expressions sns aws hive risk management python data management api gateway scala microsoft azure kinesis aws lambda data engineering sql aws glue data modeling spark kafka hadoop sqoop etl big data

Work Mode

Work from Office

Job Type

Full Time

Job Description

About the Role

Responsibilities

Design and implement scalable and secure cloud integration solutions using AWS services such as AWS Lambda, API Gateway, AWS Glue, and Amazon S3, alongside Confluent Kafka for data streaming and Databricks for data processing and analytics.
Develop and maintain integration architectures, data flows, and APIs to connect various cloud and on-premises systems, leveraging Confluent Kafka for real-time data processing or other integration tools for batch processing.
Design and implement scalable data solutions on AWS, ensuring optimal performance, reliability and cost effectiveness while adhering to best practices in cloud data management.
Develop comprehensive cloud integration architectures using AWS services like API Gateway, Lambda, SQS, SNS, Kinesis, App Flow and other relevant components to meet specific integration requirements.
Implement DevOps practices and tools to streamline development and deployment processes, ensuring efficient collaboration and continuous integration/continuous deployment (CI/CD) pipelines.
Design and implement scalable data solutions on AWS, ensuring optimal performance and cost-effectiveness while adhering to best practices in cloud data management.
Develop and Optimize ETL pipelines and data models using the Medallion Architecture (Bronze, Silver, Gold layers) for improved data processing and facilitate data transformation.
Collaborate with business analysts, data scientists, and stakeholders to understand their data requirements and deliver tailored solutions that leverage the capabilities of the Databricks Lakehouse platform.
Stay updated on AWS services and industry trends, recommending improvements and innovations to enhance integration processes.

More Jobs at Mercer

Performance Management Lead

Mumbai, Gurugram

5 - 8 yrs

INR 4 - 8 Lacs

Engagement Manager

Mumbai, Gurugram

6 - 10 yrs

INR 8 - 13 Lacs

Principal Engineer

Gurugram

8 - 10 yrs

INR 10 - 14 Lacs

Oliver Wyman – Actuarial Intern – Life

Gurugram, Bengaluru

1 - 3 yrs

INR 2 - 3 Lacs

Associate Director - Talent Strategy Consulting

Mumbai

10 - 15 yrs

INR 35 - 40 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.