Data Engineering -Microsoft Fabric, Azure (Databricks & ADF), PySpark

Job Title: Data Engineering Senior Associate Microsoft Fabric, Azure (Databricks & ADF), PySpark

Experience: 410 Years

Location: PAN India

Job Summary:

We are looking for a skilled and experienced Data Engineer with 4-10 years of experience in building scalable data solutions on the Microsoft Azure ecosystem. The ideal candidate must have strong hands-on experience with Microsoft Fabric, Azure Databricks along with strong PySpark, Python and SQL expertise. Familiarity with Data Lake, Data Warehouse concepts, and end-to-end data pipelines is essential.

Key Responsibilities:

Requirement gathering and analysis

Design and implement data pipelines using Microsoft Fabric & Databricks

Extract, transform, and load (ETL) data from various sources into Azure Data Lake Storage

Implement data security and governance measures

Monitor and optimize data pipelines for performance and efficiency

Troubleshoot and resolve data engineering issues

Provide optimized solution for any problem related to data engineering

Ability to work with a variety of sources like Relational DB, API, File System, Realtime streams, CDC etc.

Strong knowledge on Databricks, Delta tables

Required Skills:

410 years of experience in Data Engineering or related roles.

Hands-on experience in Microsoft Fabric

Hands-on experience in Azure Databricks

Proficiency in PySpark for data processing and scripting.

Strong command over Python & SQL writing complex queries, performance tuning, etc.

Experience working with Azure Data Lake Storage and Data Warehouse concepts (e.g., dimensional modeling, star/snowflake schemas).

Hands on experience in performance tuning & optimization on Databricks & MS Fabric.

Ensure alignment with overall system architecture and data flow.

Understanding CI/CD practices in a data engineering context.

Excellent problem-solving and communication skills.

Exposure to BI tools like Power BI, Tableau, or Looker.

Good to Have:

Experienced in Azure DevOps.

Familiarity with data security and compliance in the cloud.

Experience with different databases like Synapse, SQL DB, Snowflake etc.