Job
Description
We are looking for an experienced Azure Data Factory (ADF) Developer to design, develop, and optimize data integration and ETL pipelines on Azure. The ideal candidate will have strong expertise in ADF, Azure Synapse, Azure Databricks, and other Azure data services. They should be skilled in ETL processes, data warehousing, and cloud-based data solutions while ensuring performance, security, and scalability. Key Responsibilities: Design and develop ETL pipelines using Azure Data Factory (ADF) to ingest, transform, and process data. Integrate ADF with other Azure services like Azure Synapse Analytics, Azure Data Lake, Azure Databricks, and SQL Database. Develop data transformations using Mapping Data Flows, SQL, and Python/PySpark. Optimize ADF performance, data flow, and cost efficiency for scalable data solutions. Automate data pipelines, scheduling, and orchestration using triggers and event-driven workflows. Troubleshoot ADF pipeline failures, performance bottlenecks, and debugging issues. Work with Azure Monitor, Log Analytics, and Application Insights for data pipeline monitoring. Ensure data security, governance, and compliance with Azure security best practices. Collaborate with data engineers, cloud architects, and business stakeholders to define data strategies. Implement CI/CD for data pipelines using Azure DevOps, Git, and Infrastructure as Code (Terraform, ARM templates, or Bicep).
