Azure Data Engineer

Roles and Responsibilities:

Implementing highly performant, scalable and re-usable data ingestion and transformation pipelines across Azure components and services

Strong experience using Logic Apps, Functions & Data Factory

Strong experience of development using Microsoft Azure as well as experience of developing integrations for large, complex pieces of software.

Strong understanding of the technical side of CI / CD

Strong understanding of Agile

Designing and implementing a data warehouse on Azure using Azure HDInsight, Azure Data Factory, ADLS, Databricks, SQL Server, SQL DWH, Analytics Service, Event Hubs, KeyVault and other Azure services

Designing, orchestrating and implementing highly performant, scalable and re-usable data ingestion and transformation pipelines across Azure components and services

Designing and implementing Event based and streaming data ingestion and processing using Azure PaaS services

Data ingestion, data engineering and/or data curation using native Azure services or tools available in Azure Marketplace

Designing and implementing data governance, data cataloguing and data lineage solutions using tools such as Azure Data Catalog/Informatica Data Catalog

Developing physical data models in MongoDB, SQL DWH , ...etc.

Developing APIs

Have worked in agile delivery teams with DevOps ways of working to continuously deliver iterative deployments with experience in using Jira, Git Repositories, Confluence, or similar

Have had experience in data migration activities in the past including migration strategy and approach, source and target system discovery, analysis, mapping, development and reconciliation