Data Engineer

Job Overview:

We are looking for a skilled and experienced Data Engineer to join our team. The ideal candidate will have a strong background in Azure Data Factory, Databricks, Pyspark, Python , Azure SQL and other Azure cloud services, and will be responsible for building and managing scalable data pipelines, data lakes, and data warehouses . Experience with Azure Synapse Analytics, Microsoft Fabric or PowerBI will be considered a strong advantage.

Key Responsibilities:

• Design, develop, and manage robust and scalable ETL/ELT pipelines using Azure Data Factory and Databricks
• Work with PySpark and Python to transform and process large datasets
• Build and maintain data lakes and data warehouses on Azure Cloud
• Collaborate with data architects, analysts, and stakeholders to gather and translate requirements into technical solutions
• Ensure data quality, consistency, and integrity across systems
• Optimize performance and cost of data pipelines and cloud infrastructure
• Implement best practices for security, governance, and monitoring of data pipelines
• Maintain and document data workflows and architecture

Required Skills & Qualifications:

• 3–5 years of experience in Data Engineering
• Strong hands-on experience with:
  • Azure Data Factory (ADF)
  • Azure Databricks
  • Azure SQL
  • PySpark and Python
  • Azure Storage (Blob, Data Lake Gen2)
• Hands-on experience with data warehouse/Lakehouse/data lake architecture
• Familiarity with Delta Lake, MLflow, and Unity Catalog is a plus
• Good understanding of SQL and performance tuning
• Knowledge of CI/CD in Azure for data pipelines
• Excellent problem-solving skills and ability to work independently

Preferred Skills:

• Experience with Azure Synapse Analytics
• Familiarity with Microsoft Fabric
• Working knowledge of Power BI for data visualization and dashboarding
• Exposure to DevOps and infrastructure as code (IaC) in Azure
• Understanding of data governance and security best practices
• Databricks certification (e.g., Databricks Certified Data Engineer Associate/Professional)