Lead Databricks Data Engineer

**Job Description:** **Role Overview:** You will be responsible for leading the design and implementation of robust data pipelines and solutions as a Senior Databricks Data Engineer. Your expertise in Databricks, PySpark, and ADF will be crucial in ensuring optimal performance and data architecture. **Key Responsibilities:** - Design, develop, and optimize large-scale data pipelines using Databricks and PySpark. - Lead data integration efforts utilizing Azure Data Factory (ADF). - Collaborate with cross-functional teams to define data requirements and deliver effective solutions. - Ensure data quality, governance, and security are maintained across all pipelines. - Mentor junior engineers and actively contribute to architectural decisions. **Qualification Required:** - 8+ years of experience in data engineering with a strong focus on Databricks and PySpark. - Proficiency in ADF, Delta Lake, and the Azure ecosystem. - Sound knowledge of distributed computing, data modeling, and performance tuning. - Hands-on experience with CI/CD pipelines and DevOps practices for data engineering. - Excellent communication and leadership skills for effective team collaboration and decision-making. (Note: No additional details of the company were present in the provided job description),