Digital Microsoft Azure / Databricks

As a Databricks Developer, you will be responsible for designing and implementing efficient data ingestion pipelines from multiple sources, both relational and non-relational. Your role will involve integrating end-to-end data pipelines to ensure the quality and consistency of data from different source systems to target data repositories. You will also be involved in processing performance analysis and code optimization. Key Responsibilities: - Design and implement efficient data ingestion pipelines - Integrate end-to-end data pipelines - Ensure data quality and consistency - Analyze processing performance and optimize code - Implement and optimize Spark jobs, data transformations, and processing workflows - Design and implement data warehousing and ETL processes - Conduct training sessions on platform best practices - Enforce security policies and access controls - Deploy code and maintain access controls Qualifications Required: - Must have experience with Azure Databricks, Pyspark, Python, SQL - Strong knowledge of data engineering projects - Excellent communication, problem-solving, and analytical skills - Experience with Reporting tools such as PowerBI is a plus - Experience with cloud-based data platforms like Azure Data Factory/DevOps is a plus As a Databricks Developer, you will be responsible for designing and implementing efficient data ingestion pipelines from multiple sources, both relational and non-relational. Your role will involve integrating end-to-end data pipelines to ensure the quality and consistency of data from different source systems to target data repositories. You will also be involved in processing performance analysis and code optimization. Key Responsibilities: - Design and implement efficient data ingestion pipelines - Integrate end-to-end data pipelines - Ensure data quality and consistency - Analyze processing performance and optimize code - Implement and optimize Spark jobs, data transformations, and processing workflows - Design and implement data warehousing and ETL processes - Conduct training sessions on platform best practices - Enforce security policies and access controls - Deploy code and maintain access controls Qualifications Required: - Must have experience with Azure Databricks, Pyspark, Python, SQL - Strong knowledge of data engineering projects - Excellent communication, problem-solving, and analytical skills - Experience with Reporting tools such as PowerBI is a plus - Experience with cloud-based data platforms like Azure Data Factory/DevOps is a plus