Lead Data Engineer

We are seeking a highly skilled Lead Data Engineer with strong hands-on experience in AWS, Python, and PySpark. The ideal candidate will design, develop, and manage scalable data pipelines and architectures, lead a team of data engineers, and ensure data availability, reliability, and performance to support analytics, reporting, and business intelligence initiatives.

Key Responsibilities:

• Lead end-to-end data pipeline development using AWS cloud services and PySpark.
• Architect and maintain data lakes, data warehouses, and ETL workflows on AWS (e.g., S3, Glue, EMR, Redshift, Lambda).
• Design, implement, and optimize ETL/ELT pipelines using Python and PySpark for batch and real-time data processing.
• Ensure data quality, integrity, and governance across all data systems.
• Collaborate with Data Scientists, Analysts, and Product teams to understand data needs and deliver reliable data solutions.
• Manage and mentor a team of data engineers, ensuring adherence to coding standards, best practices, and version control.
• Monitor and optimize data processing performance and cost within AWS.
• Implement data security and compliance practices in alignment with organizational standards.
• Stay updated on emerging technologies in data engineering and cloud computing.

Job Type: Full-time

Pay: ₹1,000,000.00 - ₹2,300,000.00 per year