Data Engineer

Important Note (Please Read Before Applying)

Do NOT apply if:

• You have less than 10 years or more than 15 years of experience
• You lack hands-on Azure Data Engineering experience
• You are on a notice period longer than 15 days
• You do not have practical experience in Python / PySpark and Azure Data Factory

Apply ONLY if

Job Title:

Location:

Experience:

Employment Type:

Notice Period:

About the Company:

Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.

Job Overview:

Lead Data Engineer

Key Responsibilities:

1. Data Architecture & Solution Design

• Architect end-to-end Azure data solutions using

  ADF, Databricks, Synapse Analytics, and ADLS Gen2

  .
• Design modern

  Data Lakehouse

  and

  Medallion (Bronze–Silver–Gold)

  architectures.
• Define data ingestion, transformation, and storage standards across structured and unstructured datasets.
• Align architecture with enterprise data strategy, security, and compliance frameworks.
• Lead

  PoCs

  , architecture reviews, and solution design sessions.

2. Data Pipeline Development & Optimization

• Develop scalable ETL/ELT pipelines using

  Azure Data Factory

  and

  Databricks (PySpark)

  .
• Implement

  batch and real-time data ingestion

  via

  Event Hubs, Kafka, or Stream Analytics

  .
• Build optimized data layers for BI, ML, and analytics consumption.
• Apply techniques such as partitioning, caching, and indexing for performance optimization.
• Automate workflows using

  CI/CD

  and

  Infrastructure as Code (IaC)

  .

3. Data Governance & Quality Management

• Implement

  data governance, lineage, and stewardship

  frameworks.
• Establish

  data quality validation

  at ingestion and transformation levels.
• Utilize

  Azure Purview / Microsoft Fabric

  for metadata management.
• Ensure data security and compliance with policies (GDPR, HIPAA, etc.) using

  RBAC, AAD, and Key Vault

  .
• Drive

  data observability and proactive monitoring

  practices.

4. Technical Leadership & Team Management

• Lead and mentor a team of data engineers, promoting technical excellence and growth.
• Review and approve solution designs, code, and data models for quality assurance.
• Collaborate with architects, scientists, and BI teams for cohesive data delivery.
• Establish

  best practices, coding standards, and reusable frameworks

  .
• Serve as the technical SME for all Azure-based data engineering initiatives.

5. Business Collaboration & Stakeholder Alignment

• Partner with business teams to translate requirements into scalable solutions.
• Work with cloud architects to optimize performance and cost of Azure workloads.
• Contribute to

  data strategy, technology evaluation, and roadmap planning

  .
• Present technical insights clearly to senior and non-technical stakeholders.

Core Technical Skills

Azure Data Services:

• Azure Data Factory (ADF) – pipelines, data flows, triggers
• Azure Databricks – PySpark, Delta Lake, Unity Catalog
• Azure Synapse Analytics – dedicated & serverless SQL pools
• Azure Data Lake Storage Gen2 – data partitioning, access control
• Event Hubs / Kafka / Stream Analytics – real-time streaming
• Azure Purview / Microsoft Fabric – metadata & lineage tracking
• Azure Functions / Logic Apps – event-driven orchestration

Programming & Tools:

• Languages:

  Python (mandatory), SQL, Scala

• Frameworks:

  PySpark, Pandas, dbt (optional)

• Version Control:

  Git, GitHub, Azure DevOps

• CI/CD:

  Azure DevOps Pipelines, GitHub Actions

• IaC:

  Terraform, Bicep, ARM Templates

• Data Modeling:

  Star/Snowflake schema, Data Vault 2.0

Databases & Integration:

• Azure SQL DB, Cosmos DB, PostgreSQL, Oracle, SAP, and other external sources
• Strong understanding of

  data warehouse vs data lake

  concepts
• Experience integrating with

  Power BI, Azure ML, or other BI/AI platforms