Azure Data Factory

Role Overview: You will be responsible for developing and maintaining scalable data processing solutions using Databricks SQL and PySpark to enhance data analysis capabilities. Collaborate with cross-functional teams to design and implement Databricks Workflows, analyze Medicare and Medicaid Claims data, ensure data integrity, and optimize existing data pipelines. Provide technical expertise, monitor workflows, document technical specifications, and stay updated with the latest advancements in Databricks and PySpark technologies. Additionally, you will collaborate with stakeholders to understand business requirements, participate in code reviews, and contribute to the development of best practices for data processing and analysis. Key Responsibilities: - Develop and maintain scalable data processing solutions using Databricks SQL and PySpark - Collaborate with cross-functional teams to design and implement Databricks Workflows - Analyze Medicare and Medicaid Claims data to identify trends and insights - Ensure data integrity and accuracy through robust data validation and quality checks - Optimize existing data pipelines to improve performance - Provide technical expertise and support to team members - Monitor and troubleshoot data processing workflows - Document technical specifications and processes - Stay updated with the latest advancements in Databricks and PySpark technologies - Collaborate with stakeholders to understand business requirements - Participate in code reviews and provide constructive feedback - Contribute to the development of best practices and guidelines for data processing and analysis - Support the company's mission by leveraging data insights to improve healthcare services Qualifications: - Possess strong expertise in Databricks SQL, Databricks Workflows, and PySpark with a proven track record of successful implementations - Exhibit excellent problem-solving skills and the ability to work independently - Show proficiency in data validation and quality assurance techniques - Have strong communication skills to effectively collaborate with cross-functional teams and stakeholders - Display a commitment to continuous learning and staying abreast of industry trends - Hold a bachelor's degree in Computer Science, Information Technology, or a related field (relevant certifications are a plus) Role Overview: You will be responsible for developing and maintaining scalable data processing solutions using Databricks SQL and PySpark to enhance data analysis capabilities. Collaborate with cross-functional teams to design and implement Databricks Workflows, analyze Medicare and Medicaid Claims data, ensure data integrity, and optimize existing data pipelines. Provide technical expertise, monitor workflows, document technical specifications, and stay updated with the latest advancements in Databricks and PySpark technologies. Additionally, you will collaborate with stakeholders to understand business requirements, participate in code reviews, and contribute to the development of best practices for data processing and analysis. Key Responsibilities: - Develop and maintain scalable data processing solutions using Databricks SQL and PySpark - Collaborate with cross-functional teams to design and implement Databricks Workflows - Analyze Medicare and Medicaid Claims data to identify trends and insights - Ensure data integrity and accuracy through robust data validation and quality checks - Optimize existing data pipelines to improve performance - Provide technical expertise and support to team members - Monitor and troubleshoot data processing workflows - Document technical specifications and processes - Stay updated with the latest advancements in Databricks and PySpark technologies - Collaborate with stakeholders to understand business requirements - Participate in code reviews and provide constructive feedback - Contribute to the development of best practices and guidelines for data processing and analysis - Support the company's mission by leveraging data insights to improve healthcare services Qualifications: - Possess strong expertise in Databricks SQL, Databricks Workflows, and PySpark with a proven track record of successful implementations - Exhibit excellent problem-solving skills and the ability to work independently - Show proficiency in data validation and quality assurance techniques - Have strong communication skills to effectively collaborate with cross-functional teams and stakeholders - Display a commitment to continuous learning and staying abreast of industry trends - Hold a bachelor's degree in Computer Science, Information Technology, or a related field (relevant certifications are a plus)