Lead Engineer

About The Role

We are looking for a highly skilled and self-driven

Senior Data Engineer

to join our Data Engineering team. The ideal candidate will have deep expertise in building scalable data pipelines and solutions using

Databricks

,

AWS

,

PySpark

,

GitHub

, and

Control-M

. This role demands strong experience in

ETL project delivery

, a solid understanding of

data concepts

(including

data warehouses

and

lakehouses

), and the ability to independently own and deliver complex data engineering tasks.

Key Responsibilities

• Design, develop, and maintain robust data pipelines using PySpark on Databricks.
• Implement ETL workflows and orchestrate them using Control-M.
• Work with AWS services (e.g., S3) to manage data ingestion, transformation, and storage.
• Collaborate with cross-functional teams to understand data requirements and translate them into scalable solutions.
• Apply best practices in version control and CI/CD using GitHub.
• Ensure data quality, integrity, and security across all stages of the pipeline.
• Independently manage delivery timelines, stakeholder communication, and technical decisions.
• Contribute to architectural discussions and mentor junior engineers.
• Leverage AI assistants and automation tools to improve productivity and streamline development workflows.
  

Required Skills & Experience

• 6+ years of experience in Data Engineering with a strong focus on ETL projects.
• Hands-on experience with Databricks, AWS, PySpark, GitHub, and Control-M.
• Strong understanding of data warehouse and Lakehouse architectures.
• Proven ability to work independently and deliver high-quality solutions.
• Excellent problem-solving and communication skills.
• Experience in Agile/Scrum environments is a plus.
  

Preferred Qualifications

• Familiarity with AI assistants and their usage in development or delivery workflows.
• Databricks Certification
• Experience with Delta Lake, Apache Spark optimization, and data governance frameworks.
• Exposure to data modeling and performance tuning.
  

Location:

This position can be based in any of the following locations:Chennai

Current Guardian Colleagues: Please apply through the internal Jobs Hub in Workday