Lead Data Engineer(AWS)

Job Title: Lead Data Engineer – AWS, Python, PySpark

Location: Kochi/Thiruvananthapuram

Experience: 7+ Years

Budget: Up to 23 LPA

Notice Period: Immediate to 15 days

Job Overview

We are seeking an experienced Lead Data Engineer to design and manage a scalable data ingestion framework with strong focus on data quality, validation, and API development. The ideal candidate will have proven expertise in AWS-based big data pipelines and thrive in collaborative Agile environments.

Key Responsibilities

Architect and maintain end-to-end data ingestion frameworks using AWS services (Glue, Lambda, EMR, Step Functions, ECS, EC2).

Develop and implement data quality checks, validation routines, monitoring & alerting systems.

Design and deploy secure, high-performance RESTful APIs for seamless integration.

Collaborate with business stakeholders, data scientists, and operations teams.

Contribute to Agile development, sprint planning, CI/CD (GitLab).

Required Skills & Qualifications

5+ years in Data Engineering, with strong background in analytical platform development.

Expertise in Python, PySpark, SQL for ETL and large-scale data manipulation.

Hands-on experience with AWS services (Glue, S3, Lambda, EMR, Step Functions).

Strong knowledge of databases (Aurora, Postgres, DynamoDB, Redshift).

Proven RESTful API design & implementation experience.

Experience with CI/CD pipelines and Agile methodology.

Excellent problem-solving and communication skills.

Preferred Qualifications

Bachelor’s/Master’s in Computer Science or related field.

Experience with AWS Kinesis, Firehose, SQS.

Familiarity with data lakehouse architectures & data quality frameworks.

📌 Important Note: Only candidates who are immediately available (0–15 days notice) and match the job requirements should apply.