Manager Data Engineer - AWS

Key Responsibilities:

• Lead the design, development, and optimization of scalable and secure data pipelines using

  AWS services

  such as

  Glue, S3, Lambda, EMR

  , and

  Databricks Notebooks

  , Jobs, and Workflows.
• Oversee the development and maintenance of

  data lakes

  on

  AWS Databricks

  , ensuring performance and scalability.
• Build and manage robust

  ETL/ELT

  workflows using

  Python

  and

  SQL

  , handling both structured and semi-structured data.
• Implement distributed data processing solutions using

  Apache Spark

  /

  PySpark

  for large-scale data transformation.
• Collaborate with cross-functional teams including data scientists, analysts, and product managers to ensure data is accurate, accessible, and well-structured.
• Enforce best practices for

  data quality, governance, security

  , and

  compliance

  across the entire data ecosystem.
• Monitor system performance, troubleshoot issues, and drive continuous improvements in data infrastructure.
• Conduct code reviews, define coding standards, and promote engineering excellence across the team.
• Mentor and guide junior data engineers, fostering a culture of technical growth and innovation