Azure Data Engineer

Job Title:

Location:

Company Overview:

Newscape

Position Summary:

Senior Databricks Engineer

Key Responsibilities:

• Design, develop, and optimize large-scale

  data pipelines

  using

  Databricks

  ,

  PySpark

  , and

  Spark SQL

  .
• Implement

  Lakehouse architecture

  using

  Delta Lake

  and follow

  medallion architecture

  principles (Bronze, Silver, Gold layers).
• Develop and manage Databricks components:

  Jobs

  ,

  Delta Live Tables (DLT)

  ,

  Repos

  ,

  Unity Catalog

  , and

  Workflows

  .
• Collaborate with

  data architects

  ,

  data scientists

  , and

  business stakeholders

  to deliver scalable and maintainable data solutions.
• Ensure

  data governance

  ,

  security

  , and

  compliance

  using tools like

  Unity Catalog

  and

  Azure Purview

  .
• Build

  CI/CD pipelines

  for data projects using

  Azure DevOps

  ,

  GitHub Actions

  , or equivalent.
• Schedule and monitor workflows using

  Airflow

  ,

  Azure Data Factory

  , or

  Databricks Workflows

  .
• Perform data modeling, transformation, and loading using

  strong SQL and data warehousing concepts

  .
• Translate complex business requirements into technical implementations with clear documentation and stakeholder alignment.
  
• Provide mentorship and technical guidance to junior team members.
  

Required Qualifications:

• 6+ years

  of experience in

  data engineering

  , including 4

  + years on Databricks

  .
• Expert-level proficiency in

  Databricks Workspace

  ,

  DLT

  ,

  Jobs

  ,

  Repos

  , and

  Unity Catalog

  .
• Strong hands-on knowledge of

  PySpark

  ,

  Spark SQL

  , and optionally

  Scala

  .
• Experience with one or more major cloud platforms:

  Azure

  ,

  AWS

  , or

  GCP

  (Azure preferred).
• Solid understanding and hands-on experience with

  Delta Lake

  ,

  Lakehouse

  , and

  medallion architecture

  .
• Proven experience with

  CI/CD

  tools such as

  Azure DevOps

  ,

  GitHub

  ,

  Bitbucket

  , etc.
• Familiarity with orchestration tools like

  Apache Airflow

  ,

  ADF

  , or

  Databricks Workflows

  .
• Understanding of

  data governance

  ,

  lineage

  , and

  metadata management

  practices.
• Strong communication and collaboration skills, with the ability to interact effectively with technical and non-technical stakeholders.
  

Nice to Have:

• Experience in the

  healthcare domain

  and understanding of healthcare data standards.
• Exposure to

  machine learning workflows

  or support for

  data science teams

  .
• Certifications in

  Databricks

  ,

  Azure

  , or other cloud platforms.

What We Offer:

• Opportunity to work on cutting-edge technologies and transformative healthcare projects.
  
• Collaborative work environment with a focus on learning and innovation.
  
• Competitive salary and performance-based growth.
  
• Work-life balance with flexible engagement for high performers.
  
• Exposure to global healthcare leaders and next-gen data platforms.

Thanks & regards,

Swapnil Supe

HR Executive

+91 8233829595

swapnil.supe@newscapeconsulting.com