Lead/Senior Data Engineer

We are seeking a Lead/Senior Data Engineer with 7-12 years of experience to architect, develop, and optimize data solutions in a cloud-native environment. The role requires strong expertise in AWS Glue, PySpark, and Python with a proven ability to design scalable data pipelines and frameworks for large-scale enterprise systems. Prior exposure to financial services or regulated environments is a strong advantage.

Key Responsibilities
Design and implement secure, scalable pipelines using AWS Glue, PySpark, EMR, S3, Lambda, and other AWS services.Lead ETL development for structured and semi-structured data, ensuring high performance and reliability.Build reusable frameworks, automation tools, and CI/CD pipelines with AWS CodePipeline, Jenkins, or GitLab.Mentor junior engineers, conduct code reviews, and enforce best practices.Implement data governance practices including quality, lineage, and compliance standards.Collaborate with product, analytics, compliance, and DevOps teams to align technical solutions with business goals.Optimize workflows for cost efficiency, scalability, and speed.Prepare technical documentation and present architectural solutions to stakeholders.

Requirements
Strong hands-on experience with AWS Glue, PySpark, Python, and AWS services (EMR, S3, Lambda, Redshift, Athena).Proficiency in ETL workflows, Airflow (or equivalent), and DevOps practices.Solid knowledge of data governance, lineage, and agile methodologies.Excellent communication and stakeholder engagement skills.Financial services or regulated environment background preferred.