Azure Data Integrations Engineers

Azure Data Integrations Engineers | 7+ Yrs | Work Timings: Standard IST | 2 positions

Description

• Proven track record building production-grade pipelines using ADF, Synapse Pipelines, and Azure Data Lake Gen2
• Demonstrated expertise integrating with authenticated REST APIs (OAuth2, bearer tokens, pagination) and ingesting structured/semi-structured data (JSON, CSV)
• Consistently delivers in structured, async environments — with clean Jira hygiene, well-documented logic, and precise written communication
• Azure Data Integration Engineer with deep hands-on experience building Azure-native data pipelines to support enterprise-scale data ingestion into a governed Azure Data Lake environment. This is a delivery-focused role responsible for building and maintaining production-ready pipelines that move structured and semi-structured data into central lake zones from internal and external systems.
• Work closely with the engineering team to design pipeline logic, implement robust orchestration, and ensure production readiness through strong parameterization, observability, and documentation. The right candidate brings strong ownership, communicates with precision, and delivers consistently in asynchronous environments.

Responsibilities

• Design and implement scalable ingestion pipelines using Azure Data Factory, Synapse Pipelines, and Azure Data Lake Storage Gen2
• Build modular, parameterized pipeline components - including linked services, datasets, and triggers - to support batch, file-based, and API-based ingestion patterns
• Integrate data from various sources using REST APIs (OAuth2, bearer tokens, pagination), SFTP, and structured file formats (CSV, JSON)
• Define ingestion logic across raw, curated, and conformed zones within a governed enterprise lakehouse model
• Establish and maintain monitoring, logging, and alerting using Azure-native observability tools
• Own the full pipeline lifecycle - design, development, deployment, documentation, and issue resolution
• Refactor and stabilize legacy pipelines where needed to improve maintainability and performance
• Document technical logic, runtime behavior, data mapping assumptions, and schema expectations using Confluence or Markdown
• Track tasks and sprint deliverables using tools like Jira, and participate in structured status updates and reviews
• Operate with high autonomy and accountability - consistently delivering work with minimal oversight and strong async communication
• Qualifications
• 7+ years of experience in data engineering, backend integration, or cloud pipeline development

Strong expertise with:

• Azure Data Factory (ADF) - pipelines, integration runtimes, parameterization, failure handling
• Synapse Pipelines
• Azure Data Lake Gen2
• Proficiency in SQL and Python for data transformations, validation, and control logic
• Experience ingesting data from authenticated REST APIs, handling pagination, retries, and token management
• Familiarity with enterprise ingestion patterns - file lifecycle management, schema evolution, secure access layers
• Comfortable with Git-based development, pull requests, and CI/CD deployment workflows
• Able to maintain reliable pipeline observability - alerts, logging, monitoring, and root cause tracing
• Strong written communication skills and documentation habits
• Full-time availability during India Standard Time (IST)

Preferred

• Experience with metadata frameworks, ingestion registries, or data cataloging
• Background working in distributed enterprise environments or regulated data contexts
• Microsoft certification (e.g. DP-203) is a plus