Lead II - Data Engineering BS

5 years

Pune

Posted:15 hours ago

Job Description

Role Description

Responsibilities * Develop and maintain real-time data pipelines using Apache Kafka (MSK or Confluent) and AWS services. * Configure and manage Kafka connectors, ensuring seamless data flow and integration across systems. * Demonstrate a strong understanding of the Kafka ecosystem, including producers, consumers, brokers, topics, and schema registry. * Design and implement scalable ETL/ELT workflows to process large volumes of data efficiently. * Optimize data lake and data warehouse solutions using AWS services such as Lambda, S3, and Glue. * Implement robust monitoring, testing, and observability practices to ensure data platform reliability and performance. * Uphold data security, governance, and compliance standards across all data operations. Requirements * Minimum of 5 years of experience in data engineering or related roles. * Proven expertise with Apache Kafka and the AWS data stack (MSK, Glue, Lambda, S3, etc.). * Proficient in coding with Python, SQL, and Java (Java strongly preferred). Person needs to flexible to write code in Python/Java * Experience with infrastructure-as-code tools (e.g. CloudFormation) and CI/CD pipelines. * Excellent problem-solving skills and strong communication and collaboration abilities.

Skills

Python, SQL, AWS CloudFormation, CI/CD

Skills Required

Data Engineering Apache Kafka AWS Data Flow ETL Data Lake Data Warehouse Data Security Python SQL Java

Learn the skills this job asks for

Free courses, interview questions and practice for each skill.

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
UST logo
UST

IT Services and IT Consulting

Aliso Viejo CA

RecommendedJobs for You