Data Engineer

We are looking for an experienced Data Engineer with strong expertise in Databricks and Azure Data Factory (ADF) to design, build, and manage scalable data pipelines and integration solutions. The ideal candidate will have a solid background in big data technologies, cloud platforms, and data processing frameworks to support enterprise-level data transformation and analytics initiatives.

• Design, develop, and maintain robust data pipelines using

  Azure Data Factory

  and

  Databricks

  .
• Build and optimize data flows and transformations for structured and unstructured data.
• Develop scalable ETL/ELT processes to extract data from various sources including SQL, APIs, and flat files.
• Implement data quality checks, error handling, and performance tuning of data pipelines.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Work with Azure services such as

  Azure Data Lake Storage (ADLS)

  ,

  Azure Synapse Analytics

  , and

  Azure SQL

  .
• Participate in code reviews, version control, and CI/CD processes.
• Ensure data security, privacy, and compliance with governance standards.
• Strong hands-on experience with

  Azure Data Factory

  and

  Azure Databricks

  (Spark-based development).
• Proficiency in

  Python

  ,

  SQL

  , and

  PySpark

  for data manipulation.
• Experience with

  Delta Lake

  ,

  data versioning

  , and

  streaming/batch data processing

  .
• Working knowledge of

  Azure services

  such as ADLS, Azure Blob Storage, and Azure Key Vault.
• Familiarity with

  DevOps

  ,

  Git

  , and CI/CD pipelines in data engineering workflows.
• Strong understanding of data modeling, data warehousing, and performance tuning.
• Excellent analytical, communication, and problem-solving skills.