Job Responsibilities:
Design and Develop Data Pipelines:
Develop and optimise scalable data pipelines using Microsoft Fabric
, including Fabric Notebooks
, Dataflows Gen2
, Data Pipelines
, and Lakehouse architecture
. Work on both batch and real-time ingestion and transformation. Integrate with Azure Data Factory
or Fabric-native orchestration for smooth data flow.
Fabric Data Platform Implementation:
Collaborate with data architects and engineers to implement governed Lakehouse models
in Microsoft Fabric (OneLake)
. Ensure data solutions are performant, reusable, and aligned with business needs and compliance standards.
Data Pipeline Optimisation:
Monitor and improve performance of data pipelines and notebooks in Microsoft Fabric. Apply tuning strategies to reduce costs, improve scalability, and ensure reliable data delivery across domains.
Collaboration with Cross-functional Teams:
Work closely with BI developers, analysts, and data scientists to gather requirements and build high-quality datasets. Support self-service BI
initiatives by developing well-structured datasets and semantic models in Fabric.
Documentation and Reusability:
Document pipeline logic, lakehouse architecture, and semantic layers clearly. Follow development standards and contribute to internal best practices for Microsoft Fabric-based solutions.
Microsoft Fabric Platform Execution:
Use your experience with Lakehouses
, Notebooks
, Data Pipelines
, and Direct Lake
in Microsoft Fabric to deliver reliable, secure, and efficient data solutions that integrate with Power BI
, Azure Synapse
, and other Microsoft services.
Required Skills and Qualifications:
-
5+ years
of experience in data engineering within the Azure ecosystem
, with relevant hands-on experience in Microsoft Fabric
, including Lakehouse
, Dataflows Gen2
, and Data Pipelines
. - Proficiency in building and orchestrating pipelines with
Azure Data Factory
and/or Microsoft Fabric Dataflows Gen2
. - Solid experience with
data ingestion
, ELT/ETL development
, and data transformation
across structured and semi-structured sources. - Strong understanding of
OneLake
architecture and modern data lakehouse patterns
. - Strong command of
SQL,Pyspark, Python
applied to both data integration and analytical workloads. - Ability to collaborate with cross-functional teams and translate data requirements into scalable engineering solutions.
- Experience in
optimising pipelines
and managing compute resources for cost-effective
data processing in Azure/Fabric.
Preferred Skills:
- Experience working in the
Microsoft Fabric ecosystem
, including Direct Lake
, BI integration
, and Fabric-native orchestration features. - Familiarity with
OneLake
, Delta Lake
, and Lakehouse
principles in the context of Microsoft s modern data platform. - expert knowledge of
PySpark
, strong SQL
, and Python scripting
within Microsoft Fabric or Databricks notebooks. - Understanding of
Microsoft Purview
or Unity Catalog
, or Fabric-native tools for metadata
, lineage
, and access control
. - Exposure to
DevOps practices
for Fabric and Power BI, including Git integration
, deployment pipelines, and workspace governance. - Knowledge of
Azure Databricks
for Spark-based transformations and Delta Lake
pipelines is a plus.