Job
Description
We are seeking a highly skilled and motivated Azure Data Engineer to join our growing data team. In this role, you will be responsible for designing, developing, and maintaining scalable and robust data pipelines and data solutions within the Microsoft Azure ecosystem. You will work closely with data scientists, analysts, and business stakeholders to understand data requirements and translate them into effective data architectures. The ideal candidate will have a strong background in data warehousing, ETL/ELT processes, and a deep understanding of Azure data services. Responsibilities: Design, build, and maintain scalable and efficient data pipelines using Azure Data Factory, Azure Synapse Analytics, Azure Databricks, or other relevant Azure services. Develop and optimize data ingestion processes from various source systems (on-premises, cloud, third-party APIs) into Azure data platforms. Implement data warehousing solutions, including dimensional modeling and data lake strategies, using Azure Synapse Analytics, Azure Data Lake Storage Gen2, or Azure SQL Database. Write, optimize, and maintain complex SQL queries, stored procedures, and data transformation scripts. Develop and manage data quality checks, data validation processes, and data governance policies. Monitor and troubleshoot data pipeline issues, ensuring data accuracy and availability. Collaborate with data scientists and analysts to support their data needs for reporting, analytics, and machine learning initiatives. Implement security best practices for data storage and access within Azure. Participate in code reviews, contribute to architectural discussions, and promote best practices in data engineering. Stay up-to-date with the latest Azure data technologies and trends, proposing and implementing improvements where applicable. Document data flows, architectures, and operational procedures. Qualifications: Required: Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. 3 to 5 years of professional experience as a Data Engineer, with a strong focus on Microsoft Azure data platforms. Proven experience with Azure Data Factory for orchestration and ETL/ELT. Solid understanding and hands-on experience with Azure Synapse Analytics (SQL Pool, Spark Pool) or Azure SQL Data Warehouse. Proficiency in SQL and experience with relational databases. Experience with Azure Data Lake Storage Gen2. Familiarity with data modeling, data warehousing concepts (e.g., Kimball methodology), and ETL/ELT processes. Strong programming skills in Python or Spark (PySpark). Experience with version control systems (e.g., Git). Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills.
