Associate Process Manager

Role Overview:

Key Responsibilities:

• Design, develop, and optimize

  complex data pipelines and lakehouse architectures

  .
• Lead development of

  batch and streaming ingestion frameworks

  using

  ADF

  ,

  Databricks

  , and

  Event Hubs

  .
• Write

  optimized PySpark/SQL transformations

  for performance, scalability, and cost efficiency.
• Build

  reusable data frameworks and automation scripts

  for continuous integration and deployment.
• Establish

  unit testing, code review, and versioning standards

  across engineering teams.
• Integrate with

  DevOps CI/CD pipelines

  for automated releases and environment management.
• Mentor junior engineers on coding practices and architecture design.
• Collaborate with data architects on

  Fabric-based Lakehouse implementations

  .

Skills & Qualifications:

• 69 years of strong hands-on experience in

  data engineering and development

  .
• Expertise in:

  • PySpark, SQL, Python

    , and performance tuning of large-scale transformations.

  • Databricks

    ,

    ADF

    ,

    ADLS

    ,

    Fabric Data Pipelines

    , and

    Delta Lake

    .
• Deep understanding of

  data architecture

  ,

  schema design

  ,

  job orchestration

  , and

  data governance

  .
• Strong knowledge of

  CI/CD (Azure DevOps)

  and

  infrastructure-as-code

  for data pipelines.
• Exposure to

  GCP BigQuery, Dataproc, or Dataflow

  as secondary platforms.

Preferred Certifications:

• Databricks Certified Data Engineer Professional

• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)

• Microsoft Certified: Azure Solutions Architect Expert (AZ-305)

• Google Cloud Certified Professional Data Engineer