Azure Data Engineer

Job Title:

About the Role

Azure Data Engineer

Key Responsibilities

• Design, develop, and optimize scalable

  ETL/ELT pipelines

  to ingest, transform, and load healthcare data from multiple sources into data warehouses/lakes.
• Build and maintain

  robust data models

  (e.g., star schema, snowflake schema) to support reporting, analytics, and ML initiatives.
• Write

  optimized SQL queries

  and

  Python scripts

  for data cleansing, validation, transformation, and automation.
• Ensure

  data integrity, quality, and governance

  , maintaining compliance with

  HIPAA and other regulatory requirements

  .
• Collaborate with

  data scientists, ML engineers, and business stakeholders

  to understand data requirements and deliver effective solutions.
• Monitor and troubleshoot pipeline performance, implementing enhancements for

  scalability and reliability

  .
• Automate workflows using

  orchestration tools

  and integrate with

  CI/CD pipelines

  .
• Contribute to evaluating and implementing

  new data tools and technologies

  .
• Maintain thorough documentation of

  data pipelines, models, and flows

  .

Required Technical Skills

• Programming:

  Strong proficiency in

  Python

  (data processing, automation).

• Databases:

  Advanced

  SQL expertise

  , with hands-on experience in relational and NoSQL databases.

• ETL/ELT:

  Proven experience designing and implementing scalable pipelines.

• Azure Cloud:

  Hands-on experience with

  Azure Data Lake, Azure Synapse Analytics, Azure Data Factory

  , and related services.

• Containerization:

  Familiarity with

  Docker

  for deploying data applications.

• Version Control:

  Proficient with

  Git/GitHub

  for code management.

• Testing:

  Experience using

  unittest / Pytest

  frameworks for pipeline validation.

• CI/CD:

  Working knowledge of

  continuous integration and deployment

  in Azure environments.

• Data Modeling:

  Strong understanding of

  data warehousing concepts and schema design

  .