Senior Data Engineer

About the Project:

We are engaging with one of customer to build and operationalize their Enterprise Data Platform (EDP). It is a strategic initiative to mature their data and analytics capabilities across the organization. The project involves building out a robust data platform, establishing a new operating model, and delivering foundational data products. You will be working closely with the customer to fill critical gaps in architecture, governance, security, and data product delivery.

What You'll Do:

As a Data Engineer on this team, you will be a hands-on contributor across all facets of the project. Your responsibilities will be directly mapped to the customer's needs and gaps, including:

• Platform Foundations (Reference Architecture & DataOps):

  Collaborate on the design and implementation of a consolidated EDP reference architecture. You will be responsible for building out and hardening the platform's core components using

  Azure Synapse

  ,

  Azure Databricks

  , and

  Azure Data Lake Storage (ADLS)

  . You will help define and implement a

  DataOps

  model, including CI/CD pipelines, environments, and observability.

• Data Governance & Quality:

  Support the operationalization of a data governance framework. This includes implementing data classification policies, setting up

  data quality scorecards

  , and configuring a data catalog with ownership and lineage in a tool like

  Azure Purview

  .

• Data Security & Access:

  Implement fine-grained access controls using patterns like

  Row-Level Security

  and

  Column-Level Security

  . You'll help design and apply a security model using

  Azure Entra ID groups

  and implement

  policy-as-code

  to protect PII.

• Data Product Pipeline Factory:

  Build and maintain

  scalable and repeatable data pipelines

  to ingest, transform, and publish data from various sources. You will work on creating a standardized framework for data products (from bronze to gold layers), ensuring observability and adherence to quality standards.

• Data Catalog & Glossary:

  Configure and populate the enterprise data catalog with metadata and a business glossary. You will be responsible for migrating "priority" datasets and embedding governance checks directly into the SDLC.

• Data Literacy & Change Management:

  Contribute to the enablement of data stewards, engineers, and analysts by providing practical, role-based training anchored to the data products you build.

What We're Looking For:

We are seeking hands-on data engineers with a passion for building robust, well-governed, and secure data platforms. A strong candidate for this team will have:

• Deep Azure Data Services Expertise:

  Proven, hands-on experience with the Azure data ecosystem, specifically

  Azure Synapse Analytics

  ,

  Azure Databricks

  ,

  Azure Data Lake Storage (ADLS)

  , and

  Azure Purview

  . Experience with

  Unity Catalog

  is highly desirable.
• Solid experience with Azure data services, including

  Azure Data Lake Storage (ADLS)

  and

  Azure Synapse Analytics

  .

• Data Engineering Fundamentals:

  Strong background in data modeling, ETL/ELT processes, and building scalable data pipelines for batch and streaming data.

• Programming & Scripting:

  Proficiency in

  Python

  and

  PySpark

  is essential for data transformation and pipeline development in Azure Databricks. Strong

  SQL

  skills are also required.

• DevOps & CI/CD:

  Experience with

  DataOps

  principles, including setting up and maintaining CI/CD pipelines using tools like Azure DevOps or GitHub Actions. Familiarity with managing environments and infrastructure as code is a plus.

• Data Governance & Security:

  Practical experience or a solid understanding of data governance concepts, including data quality, lineage, metadata management, and access control models (

  RBAC/ABAC

  ).
• Knowledge of data modeling techniques, data warehousing, and lakehouse architectures.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

• 5+ years of experience in a data engineering role.

• Excellent problem-solving, analytical, and communication skills.
• Demonstrated ability to work effectively in a collaborative and fast-paced environment.