Job
Description
As an Azure Data Engineer, you will be responsible for integrating, transforming, and consolidating data from various structured and unstructured sources using tools such as Azure Synapse Analytics and Azure Data Factory. Your role will involve building robust data pipelines, implementing data transformation processes, and supporting data structures for optimal extraction and loading of data. Additionally, you will work on processing data using Synapse Analytics/Databricks notebooks and understanding customer needs. - Create Synapse Analytics/Databricks notebook to process data. - Integrate and consolidate data from multiple sources and load to Azure Data Lake Storage. - Identify, design, and implement internal process improvements such as automating manual processes and optimizing data delivery. - 3+ years of relevant experience in Pyspark and Azure Synapse Analytics/Azure Databricks. - Proficiency in integrating, transforming, and consolidating data from various structured and unstructured data sources. - Strong experience in implementing Synapse Analytics/Databricks notebooks using Python/Pyspark. - Good experience in Azure Data Factory, Azure Data Lake Storage, Storage Services, Serverless architecture, and Azure functions. - Advanced working knowledge in building robust data pipelines using Azure Data Factory. - Experience in working with SQL or native SQL query languages. - Exposure to basic issues in working within a Cloud (Azure) environment. - Experience in working in an Agile project management environment. - Good to have experience in SSIS/ETL transformation processes. Please Note: Exposure to Jira and Confluence, advanced working SQL knowledge, experience working with relational/non-Relational databases, and exposure to cloud-based Datawarehouse are additional qualifications that would be beneficial for this role. As an Azure Data Engineer, you will be responsible for integrating, transforming, and consolidating data from various structured and unstructured sources using tools such as Azure Synapse Analytics and Azure Data Factory. Your role will involve building robust data pipelines, implementing data transformation processes, and supporting data structures for optimal extraction and loading of data. Additionally, you will work on processing data using Synapse Analytics/Databricks notebooks and understanding customer needs. - Create Synapse Analytics/Databricks notebook to process data. - Integrate and consolidate data from multiple sources and load to Azure Data Lake Storage. - Identify, design, and implement internal process improvements such as automating manual processes and optimizing data delivery. - 3+ years of relevant experience in Pyspark and Azure Synapse Analytics/Azure Databricks. - Proficiency in integrating, transforming, and consolidating data from various structured and unstructured data sources. - Strong experience in implementing Synapse Analytics/Databricks notebooks using Python/Pyspark. - Good experience in Azure Data Factory, Azure Data Lake Storage, Storage Services, Serverless architecture, and Azure functions. - Advanced working knowledge in building robust data pipelines using Azure Data Factory. - Experience in working with SQL or native SQL query languages. - Exposure to basic issues in working within a Cloud (Azure) environment. - Experience in working in an Agile project management environment. - Good to have experience in SSIS/ETL transformation processes. Please Note: Exposure to Jira and Confluence, advanced working SQL knowledge, experience working with relational/non-Relational databases, and exposure to cloud-based Datawarehouse are additional qualifications that would be beneficial for this role.
