Azure Data Engineer

Job Description

Role & responsibilities Job Summary: We are looking for a Senior Data Engineer with strong expertise in Azure Data Services, Python, and data orchestration tools to join our team. The ideal candidate should have hands-on experience with Azure Data Factory (ADF), Azure Synapse, Dagster or Apache Airflow, and Azure DevOps Pipelines. Strong communication and presentation skills are a must, as this role involves collaborating with stakeholders, presenting data solutions, and ensuring smooth data pipeline operations. This role requires a problem-solver with a deep understanding of data engineering best practices, cloud-based ETL processes, and modern data platforms. If you are passionate about building scalable data solutions and driving innovation, wed love to hear from you! Key Responsibilities: Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF), Azure Synapse, and Python. Implement and manage data orchestration workflows using Dagster or Apache Airflow. Develop and optimize ETL/ELT processes for efficient data ingestion, transformation, and storage. Ensure seamless integration of structured and unstructured data from multiple sources. Work closely with data scientists, analysts, and business teams to understand data requirements and provide scalable solutions. Implement Azure DevOps Pipelines for continuous integration and deployment of data engineering solutions. Monitor and optimize data pipelines for performance, cost, and reliability. Troubleshoot and resolve data pipeline failures, latency issues, and bottlenecks. Maintain high standards of data security, governance, and compliance. Communicate complex data solutions in a clear and effective manner to both technical and non-technical stakeholders. Provide technical leadership and mentorship to junior team members. Stay updated with the latest trends in cloud data engineering and modern data platforms. Required Skills & Experience: Azure Data Services – Strong hands-on experience with Azure Data Factory (ADF), Azure Synapse Analytics (SQL Pools, Spark Pools), and Azure Storage. Python for Data Engineering – Proficiency in writing efficient, scalable data processing scripts. Data Orchestration – Experience working with Dagster or Apache Airflow for scheduling and monitoring data workflows. Azure DevOps Pipelines – Expertise in CI/CD pipelines for automated deployment of data solutions. ETL/ELT Development – Strong understanding of data transformation, ingestion strategies, and performance tuning. Data Modeling & SQL – Experience in designing and optimizing data models for analytics and reporting. Big Data & Distributed Computing (Preferred) – Exposure to Apache Spark, Databricks, or other big data frameworks. Data Governance & Security – Knowledge of RBAC, encryption, and compliance best practices. Excellent Communication & Presentation Skills – Ability to explain technical concepts to business users and present data-driven solutions effectively. Preferred Qualifications: Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field. Azure Certifications (DP-203, AZ-104, AZ-400, or equivalent) are a plus. Experience with Terraform, Bicep, or Infrastructure as Code (IaC) for managing cloud resources.