Data Engineer

Key Responsibilities

• Design scalable ETL/ELT data pipelines using Azure Data Factory, Synapse Pipelines, and Databricks.
• Develop data ingestion workflows for structured and semi-structured data sources, including IoT streams and batch files.
• Define and enforce data modeling standards for the raw, curated, and semantic layers in ADLS Gen2.
• Create PySpark jobs and reusable transformation frameworks for cleansing, validation, and enrichment.
• Oversee data partitioning, versioning, and metadata strategies to ensure high performance and maintainability.
• Coordinate with ML engineers to support feature engineering pipelines and training datasets.
• Mentor data engineers, conduct code reviews, and enforce best practices in CI/CD workflows.
• Implement monitoring and alerting for data pipelines, including performance metrics and SLA tracking.

Primary Skills and Technologies

Azure Data Factory, Azure Databricks, Synapse Analytics, PySpark, Data Lake Gen2, ETL Frameworks, Data Modeling, Azure DevOps for Data CI/CD, Delta Lake.

Experience and Qualifications

• 5+ years in data engineering roles on Azure data platforms.
• Experience building large-scale data solutions processing real-time and batch workloads.
• Strong understanding of data governance, quality, and security practices.
• Microsoft Azure Data Engineer Associate certification is a plus.