Azure Data Engineer

A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics. We focus on a collection of organisational technology capabilities, including business intelligence, data management, and data assurance that help our clients drive innovation, growth, and change within their organisations to keep up with the changing nature of customers and technology. We make impactful decisions by mixing mind and machine to leverage data, understand and navigate risk, and help our clients gain a competitive edge.

Responsibilities:

• Design, develop, and maintain end-to-end data pipelines and data integration solutions using Azure Data Factory and Microsoft Fabric.
• Build and optimize Lakehouse and Data Warehouse solutions within Microsoft Fabric.
• Develop data transformation logic using PySpark, Spark SQL, and Python.
• Implement medallion architecture (Bronze, Silver, Gold) for data lake solutions.
• Create and manage data models, semantic models, and datasets for reporting and analytics.
• Ensure data quality, consistency, and security across all data platforms.
• Monitor, troubleshoot, and optimize data pipeline performance.
• Collaborate with data architects, data scientists, analysts, and business stakeholders to understand data requirements.
• Implement data governance policies and best practices.
• Document technical designs, data flows, and solution architectures.
• Stay updated with the latest Azure and Microsoft Fabric features and best practices.

Mandatory skill sets:

• 5-10 years of hands-on experience in data engineering and analytics.
• Proficiency in Azure Data Services including:
• Azure Data Factory
• Azure Synapse Analytics
• Azure Data Lake Storage (ADLS Gen2)
• Azure SQL Database
• Proficiency in Microsoft Fabric workloads:
• Data Engineering (Lakehouse, Notebooks)
• Data Warehouse
• Data Pipelines
• OneLake
• Strong programming skills in PySpark, Spark SQL, Python, and SQL.
• Experience with data modeling (dimensional modeling, star/snowflake schemas).
• Knowledge of ETL/ELT design patterns and best practices.
• Experience with Delta Lake and lakehouse architecture.
• Strong analytical, problem-solving, and troubleshooting skills.
• Excellent communication and collaboration skills.

Preferred skill sets:

• Microsoft Certified: Azure Data Engineer Associate (DP-203).
• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600).
• Experience with Databricks and Delta Lake.
• Experience with Power BI for data visualization and reporting.
• Knowledge of CI/CD pipelines for data engineering (Azure DevOps, Git).
• Experience working in Agile/Scrum environments.
• Exposure to data quality frameworks and tools.

Years of experience required:

• 5-10years

Education qualification:

• BE, B.Tech, ME, M,Tech, MBA, MCA (60% above)