Lead Data Engineer

Description

• Lead the design, development, and implementation of scalable data pipelines and ELT processes using Databricks, DLT, dbt, Airflow, and other tools.




• Collaborate with stakeholders to understand data requirements and deliver high-quality data solutions.




• Optimize and maintain existing data pipelines to ensure data quality, reliability, and performance.




• Develop and enforce data engineering best practices, including coding standards, testing, and documentation.




• Mentor junior data engineers, providing technical leadership and fostering a culture of continuous learning and improvement.




• Monitor and troubleshoot data pipeline issues, ensuring timely resolution and minimal disruption to business operations.




• Stay up to date with the latest industry trends and technologies, and proactively recommend improvements to our data engineering practices.

Qualifications

• Systems (MIS), Data Science or related field.




• 15 years of experience in data engineering and/or architecture, with a focus on big data technologies.




• Extensive production experience with Databricks, Apache Spark, and other related technologies.




• Familiarity with orchestration and ELT tools like Airflow, dbt, etc.




• Expert SQL knowledge.




• Proficiency in programming languages such as Python, Scala, or Java.




• Strong understanding of data warehousing concepts.




• Experience with cloud platforms such as Azure, AWS, Google Cloud.




• Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.




• Strong communication and leadership skills, with the ability to effectively mentor and guide




• Experience with machine learning and data science workflows




• Knowledge of data governance and security best practices




• Certification in Databricks, Azure, Google Cloud or related technologies.

Job

Primary Location

Schedule:

Travel:

Req ID:

Job Hire Type