Data Engineer - Azure

Job Description

Role Summary We are seeking skilled Data Engineers with prior experience on Azure data engineering services and SQL server to join our team. As a Data Engineer, you will be responsible for designing and implementing robust data infrastructure, building scalable data pipelines, and ensuring efficient data integration and storage. Responsibilities Design, develop, and maintain scalable data pipelines and use Azure Data Factory and Azure Stream Analytics Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness Implement data governance and security best practices to ensure compliance and data integrity Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities Qualifications Proven experience as a Data Engineer or in a similar role Experience in designing and hands-on development in cloud-based (AWS/Azure) analytics solutions Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, Azure App Service, Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential Experience in SQL server and procedures Thorough understanding of Azure Infrastructure offerings Strong experience in common data warehouse modelling principles including Kimball, Inmon Experience in additional Modern Database terminologies Working knowledge of Python or Java or Scala is desirable Strong knowledge of data modelling, ETL processes, and database technologies Experience with big data processing frameworks (Hadoop, Spark) and data pipeline orchestration tools (Airflow) Solid understanding of data governance, data security, and data quality best practices Strong analytical and problem-solving skills, with attention to detail