Lead Data Engineer

As a Lead Data Engineer specializing in Databricks, your role will involve designing, developing, and optimizing scalable, efficient, and secure data solutions. You will collaborate with cross-functional teams to analyze data requirements and translate them into robust data architectures. Your responsibilities will include developing and maintaining ETL pipelines using Databricks and integrating with Azure Data Factory, implementing machine learning models, and ensuring data quality, governance, and security practices. Key Responsibilities: - Lead the design, development, and optimization of data solutions using Databricks - Collaborate with cross-functional teams to gather and analyze data requirements - Develop and maintain ETL pipelines leveraging Databricks and integrating with Azure Data Factory - Implement machine learning models and advanced analytics solutions - Ensure data quality, governance, and security practices are adhered to - Provide technical leadership and mentorship to junior engineers - Stay updated on the latest trends in data engineering, Databricks, Generative AI, and Azure Data Factory Qualifications Required: - Bachelors or masters degree in computer science, Information Technology, or a related field - 7+ to 10 years of experience in data engineering with a focus on Databricks - Proven expertise in building and optimizing data solutions using Databricks - Proficiency in SQL and programming languages such as Python or Scala - Strong understanding of data modeling, ETL processes, and Data Warehousing/Data Lakehouse concepts - Familiarity with cloud platforms, particularly Azure, and technologies like Docker - Excellent analytical, problem-solving, and communication skills - Demonstrated leadership ability with experience mentoring junior team members Additional Details: You will have the opportunity to work with Generative AI technologies and cloud platforms like AWS or GCP. Knowledge of data governance frameworks and tools will be advantageous in this role. (Note: Company details were not provided in the job description.) As a Lead Data Engineer specializing in Databricks, your role will involve designing, developing, and optimizing scalable, efficient, and secure data solutions. You will collaborate with cross-functional teams to analyze data requirements and translate them into robust data architectures. Your responsibilities will include developing and maintaining ETL pipelines using Databricks and integrating with Azure Data Factory, implementing machine learning models, and ensuring data quality, governance, and security practices. Key Responsibilities: - Lead the design, development, and optimization of data solutions using Databricks - Collaborate with cross-functional teams to gather and analyze data requirements - Develop and maintain ETL pipelines leveraging Databricks and integrating with Azure Data Factory - Implement machine learning models and advanced analytics solutions - Ensure data quality, governance, and security practices are adhered to - Provide technical leadership and mentorship to junior engineers - Stay updated on the latest trends in data engineering, Databricks, Generative AI, and Azure Data Factory Qualifications Required: - Bachelors or masters degree in computer science, Information Technology, or a related field - 7+ to 10 years of experience in data engineering with a focus on Databricks - Proven expertise in building and optimizing data solutions using Databricks - Proficiency in SQL and programming languages such as Python or Scala - Strong understanding of data modeling, ETL processes, and Data Warehousing/Data Lakehouse concepts - Familiarity with cloud platforms, particularly Azure, and technologies like Docker - Excellent analytical, problem-solving, and communication skills - Demonstrated leadership ability with experience mentoring junior team members Additional Details: You will have the opportunity to work with Generative AI technologies and cloud platforms like AWS or GCP. Knowledge of data governance frameworks and tools will be advantageous in this role. (Note: Company details were not provided in the job description.)