Azure - Lead Data Engineer

Job Description

Please share your updated profile to Monikapriya.s@htcinc.com along with below details, Total Exp : Rel Exp : Current CTC : Exp CTC : Notice Period: We are looking for a highly skilled Lead Data Engineer with expertise in Azure, Python, Databricks, and ETL to drive data engineering initiatives. The ideal candidate will have strong leadership experience, hands-on technical skills, and a deep understanding of cloud-based data solutions. Key Responsibilities: Lead the design, development, and implementation of scalable data pipelines using Azure Data Services, Databricks, and Python . Architect and optimize ETL workflows to ensure data integrity and efficiency. Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and provide robust solutions. Manage and mentor a team of data engineers, providing guidance on best practices and performance optimization. Implement and enforce data governance, security, and compliance standards. Troubleshoot and resolve data pipeline issues, ensuring high availability and reliability. Optimize data storage, transformation, and retrieval for large datasets in a cloud environment. Required Skills & Experience: 8+ years of experience in Data Engineering , with at least 2+ years in a lead role . Strong hands-on experience with Azure Data Services (Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake). Expertise in Databricks and Spark for big data processing. Proficiency in Python for data manipulation and pipeline development. Deep understanding of ETL/ELT processes and best practices. Experience with CI/CD pipelines and DevOps practices for data engineering. Strong knowledge of data modeling, warehousing concepts, and performance tuning . Excellent problem-solving, analytical, and communication skills.