Senior Data Engineer - Microsoft Fabric

Role: Senior Data Engineer - Microsoft Fabric

Position Overview

Senior Data Engineer

Microsoft Fabric

Key Responsibilities:

• Design and implement scalable data pipelines and ETL/ELT processes within Microsoft Fabric from a code-first approach
• Develop and maintain notebooks, data pipelines, workspace and other Fabric item configurations
• Build and optimize data architectures using delta tables, lakehouse, and data warehouse patterns
• Implement data modelling solutions including star schema, snowflake schema, and slowly changing dimensions (SCDs)
• Performance tune Delta, Spark, and SQL workloads through partitioning, optimization, liquid clustering, and other advanced techniques
• Develop and deploy Fabric solutions using CI/CD practices via Azure DevOps
• Integrate and orchestrate data workflows using Fabric Data Agents and REST APIs
• Collaborate with development team and stakeholders to translate business requirements into technical solutions
  

Requirements

Total experience- Min 7 yrs with 3+yrs in Data Fabric

Microsoft Fabric Expertise:

• Hands-on experience with Fabric notebooks, pipelines, and workspace configuration
• Fabric Data Agent implementation and orchestration
• Fabric CLI and CI/CD deployment practices
  

Programming & Development:

• Python (advanced proficiency)
• PySpark for distributed data processing
• Pandas and Polars for data manipulation
• Experience with Python libraries for data engineering workflows
• REST API development and integration
  

Data Platform & Storage:

• Delta Lake and Iceberg table formats
• Delta table optimization techniques (partitioning, Z-ordering, liquid clustering)
• Spark performance tuning and optimization
• SQL query optimization and performance tuning
  

Development Environment:

• Visual Studio Code
• Azure DevOps for CI/CD and deployment pipelines
• Experience with both code-first and low-code development approaches
  

Data Modeling:

• Data warehouse dimensional modeling (star schema, snowflake schema)
• Slowly Changing Dimensions (SCD Type 1, 2, 3)
• Modern lakehouse architecture patterns
• Metadata driven approaches
  

Preferred Qualifications:

• 5+ years of data engineering experience
• Previous experience with large-scale data platforms and enterprise analytics solutions
• Strong understanding of data governance and security best practices
• Experience with Agile/Scrum methodologies
• Excellent problem-solving and communication skills