Area Manager-Data Engineering-Cloud Data Engineering

Responsibilities

• Design, develop, test, and deploy robust and scalable data pipelines using Azure Databricks, PySpark, and Spark SQL for processing large volumes of structured and unstructured data.
• Implement workflow orchestration using Azure Data Factory and Databricks Workflows to automate ETL/ELT processes.
• Implement and maintain CI/CD pipelines for Databricks notebooks, scripts, and configurations using Azure DevOps or GitHub Actions, promoting code across dev, test, and prod environments.
• Develop and maintain healthcare data models, including data entities, relationships, and data flows, to support healthcare analytics and decision-making.
• Design and implement enterprise-wide BI architectures. Develop and maintain data warehouses and data marts.
• Develop and maintain reports, dashboards, and data visualizations. Collaborate with stakeholders to understand business needs and provide data-driven insights
  

• 6+ years of experience as a Data Engineer with a proven track record of building data pipelines in the cloud.
• 3+ years of hands-on experience with Azure Databricks
• Strong experience with core Azure data services Azure Data Lake Storage (Gen 2), Azure data factory.
• Proficiency in Python and expert-level skills in SQL
• Solid understanding of data warehousing concepts, dimensional modeling (star/snowflake schemas), and data architecture principles
• Experience with CI/CD pipelines (Azure DevOps, Jenkins, GitHub Actions) and version control systems (Git).
• Demonstrated experience and knowledge of strategic problem solving and frameworks, and project management skills
• Excellent written and verbal communication with the ability to establish credibility and strong relationships with stakeholders