Azure Databricks Data Engineer

Role:

Work Mode:

Job Summary:

Key Responsibilities:

• Design and implement scalable, high-performance data solutions

  using Databricks on Azure to support business analytics and reporting needs.
• Collaborate with

  cross-functional teams

  (data science, BI, product, infrastructure) to integrate big data solutions with the existing IT ecosystem.
• Develop, optimize, and manage

  ETL/ELT pipelines

  , data lakes, and data warehouses ensuring efficiency and robustness.

• Perform data modeling

  , validation, and ensure data accuracy, integrity, and reliability across platforms.
• Provide subject matter expertise on

  data storage solutions

  and manage large-scale data ingestion and transformation workflows.
• Implement

  CI/CD practices

  using tools such as

  Azure DevOps, Jenkins, TFS, PowerShell

  , and automate deployment pipelines.
• Ensure adherence to

  data security, privacy policies

  , and compliance requirements.
• Mentor and guide junior engineers, lead project segments, and contribute to architectural reviews and standards.

Must-Have Skills:

• Strong hands-on experience with

  Azure Databricks

  and associated Azure cloud services (e.g., Data Lake, Synapse

  , Blob Storage

  )
• Proficiency in

  Spark, PySpark

  , and

  SQL

  for data transformation and analytics
• Solid understanding of

  big data architecture

  and

  distributed computing principles

• Experience in

  CI/CD automation

  using Azure DevOps, Jenkins, TFS, or equivalent tools
• Ability to design and manage

  large-scale data pipelines

  and complex data workflows
• Strong communication, collaboration, and problem-solving skills

Good to Have:

• Exposure to machine learning workflows on Databricks
• Familiarity with

  Delta Lake, Azure Machine Learning, or Power BI integration

• Certifications in Azure or Databricks