Microsoft fabric + Data bricks + Pyspark

Job Title:

Experience:

Location:

Employment Type:

Job Summary:

Data Engineer

Key Responsibilities:

• Requirement gathering and analysis
• Experience with different databases like Synapse, SQL DB, Snowflake etc.
• Design and implement data pipelines using Microsoft Fabric & Databricks
• Extract, transform, and load (ETL) data from various sources into Azure Data Lake Storage
• Implement data security and governance measures
• Monitor and optimize data pipelines for performance and efficiency
• Troubleshoot and resolve data engineering issues
• Provide optimized solution for any problem related to data engineering
• Ability to work with a variety of sources like Relational DB, API, File System, Realtime streams, CDC etc.
• Provide mentorship on debugging and problem-solving.
• Strong knowledge on Databricks, Delta tables

Required Skills:

• More than 8 years of experience in

  Data Engineering

  or related roles.
• Hands-on experience in

  Microsoft Fabric

• Hands-on experience in

  Azure Databricks

• Proficiency in

  PySpark

  for data processing and scripting.
• Strong command over

  Python

  &

  SQL

  – writing complex queries, performance tuning, etc.
• Experience working with

  Azure Data Lake Storage

  and

  Data Warehouse

  concepts (e.g., dimensional modeling, star/snowflake schemas).
• Hands on experience in performance tuning & optimization on Databricks & MS Fabric.
• Ensure alignment with overall system architecture and data flow.
• Understanding CI/CD practices in a data engineering context.
• Excellent problem-solving and communication skills.
• Exposure to

  BI tools

  like

  Power BI

  ,

  Tableau

  , or

  Looker

  .

Good to Have:

• Experienced in

  Azure DevOps

  .
• Knowledge of

  Scala

  or other distributed processing frameworks.
• Familiarity with

  data security

  and

  compliance

  in the cloud.
• Experience in leading a development team.