Lead Engineer

About the Company

Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.

Lead Data Engineer (L5) – Python & Azure

Job Overview:

We are seeking a highly skilled Lead Data Engineer with deep expertise in

Microsoft Azure data ecosystem to design, develop, and maintain scalable data

pipelines and architectures. The ideal candidate will play a key role in building robust

data solutions that support advanced analytics, BI, and AI workloads across the

organization. This role involves working with cross-functional teams — data scientists,

analysts, and business stakeholders — to ensure high-quality, secure, and performant

data delivery.

Total experience : 10+ years

Key Responsibilities:

1. Data Architecture & Solution Design

● Architect end-to-end data solutions on Azure, leveraging services like Azure Data

Factory, Azure Databricks, Azure Synapse Analytics, and ADLS Gen2.

● Design and implement modern data architectures, including Data Lakehouse and

Medallion (Bronze–Silver–Gold) architectures.

● Define standards for data ingestion, transformation, and storage across

structured, semi-structured, and unstructured data.

● Ensure alignment of data solutions with enterprise data strategy, security, and

compliance frameworks.

● Lead proof-of-concept (PoC) and architecture review sessions for new projects.

2. Data Pipeline Development & Optimization

● Build and maintain scalable, fault-tolerant ETL/ELT pipelines using Azure Data

Factory (ADF) and Azure Databricks (PySpark).

● Implement real-time and batch data ingestion using Azure Event Hubs, Kafka, or

Stream Analytics.

● Manage data transformations and curation layers for consumption by BI, AI/ML,

and analytics teams.

● Apply data partitioning, indexing, caching, and query tuning techniques to

optimize performance.● Automate data workflows and orchestrations using CI/CD pipelines and

infrastructure as code.

3. Data Governance & Quality Management

● Establish and enforce data governance standards, including data lineage,

metadata, and stewardship.

● Implement data quality frameworks and validation checks at ingestion and

transformation stages.

● Drive adoption of Azure Purview (or Microsoft Fabric governance) for cataloging

and classification.

● Ensure security and compliance with policies (e.g., GDPR, HIPAA) using Azure

RBAC, AAD, and Key Vault.

● Promote data observability and monitoring practices for proactive issue

detection.

4. Technical Leadership & Team Management

● Lead a team of data engineers, providing technical guidance, mentorship, and

career development.

● Review and approve solution designs, data models, and code to maintain quality

and consistency.

● Collaborate with data architects, data scientists, and BI teams to align technical

priorities.

● Define best practices, coding standards, and reusable frameworks for the data

engineering team.

● Serve as the technical point of contact for all Azure-related data initiatives.