Lead Data Engineer

As a Lead Data Engineer at our company located in Indore, Madhya Pradesh, you will be an integral part of our dynamic team, bringing your extensive experience and expertise in PySpark, Azure Data Bricks, Azure Data Factory, SQL, and Power BI to the table. Your primary responsibility will involve designing, constructing, and managing scalable data solutions that are crucial for our various projects. Your role will encompass data architecture, pipeline development, and ensuring data governance across different initiatives. Your key responsibilities will include leading the creation and enhancement of robust data pipelines utilizing Azure Data Factory and PySpark, building scalable data solutions and workflows within Azure Databricks, writing efficient SQL queries for data manipulation, and developing and maintaining dashboards and reports using Power BI. You will also be tasked with upholding best practices in data security, governance, and compliance, collaborating with diverse teams, mentoring junior engineers, and monitoring data pipelines for optimal performance. To excel in this role, you must possess a minimum of 5 years of hands-on experience in PySpark, Azure Data Factory, Azure Databricks, SQL, and at least 2 years of experience in Power BI. Additionally, you should have a solid understanding of ETL/ELT processes, modern data architecture, and expertise in handling large-scale structured and semi-structured datasets. Exposure to Microsoft Fabric for integrated analytics and automation, knowledge of data security, governance, and privacy frameworks, as well as familiarity with DevOps practices in data engineering environments are considered advantageous. Joining our team will provide you with the opportunity to work on impactful projects with cutting-edge technologies in a fast-paced, collaborative, and learning-driven environment. You can look forward to a competitive salary and a growth path within our tech-driven organization.,