Azure Data Engineer

Role Overview: As a Senior Software Engineer (SSE) specializing in Kafka, Python, and Azure Databricks, you will play a crucial role in leading and contributing to healthcare data engineering projects. Your primary responsibility will involve building scalable real-time data pipelines and processing large healthcare datasets within a secure and compliant cloud environment. Key Responsibilities: - Designing and developing scalable real-time data streaming solutions using Apache Kafka and Python. - Architecting and implementing ETL/ELT pipelines utilizing Azure Databricks for both structured and unstructured healthcare data. - Optimizing and maintaining Kafka applications, Python scripts, and Databricks workflows to ensure high performance and reliability. - Ensuring data integrity, security, and compliance with healthcare standards such as HIPAA and HITRUST. - Collaborating with data scientists, analysts, and business stakeholders to gather requirements and translate them into robust data solutions. - Mentoring junior engineers, conducting code reviews, and advocating for engineering best practices. - Remaining updated with the latest technologies in cloud computing, big data, and healthcare data standards. - Contributing to the development of CI/CD pipelines and containerized environments, including Docker and Kubernetes. Qualification Required: - At least 4 years of hands-on experience in data engineering roles. - Proficiency in Kafka, Python, Azure Databricks, SQL, NoSQL databases, and data modeling for large-scale systems. - Exposure to containerization tools like Docker, orchestration using Kubernetes, and CI/CD pipelines for data applications. - Prior experience with healthcare datasets such as EHR, HL7, FHIR, and claims data is highly desirable. Note: No additional details of the company are present in the job description.,