Principal Data Engineer

As a Principal Software Engineer for Data, the person will lead the design and implementation of scalable, secure, and high-performance data pipelines across that involves healthcare clinical data, using modern big data and cloud technologies (Azure, Databricks, and Spark), ensuring alignment with UnitedHealth Group’s data governance standards. This role requires a hands-on leader who can write and review code, mentor teams, and collaborate across business and technical stakeholders to drive data strategy and innovation. The person needs to be ready to take up AI and AIOps as part of their work and support the data science teams with ideas and review their work.

• Primary Responsibilities

Architecture

• Design and lead the implementation of robust, scalable, and secure data architectures for clinical and healthcare data for batch and real time pipelines.
• Architect end-to-end data pipelines using big data and cloud-native technologies (e.g., Spark, Databricks, Azure Data Factory).
• Ensure data solutions meet performance, scalability, and compliance requirements, including HIPAA and internal governance policies.
• Have good experience with designing, evolving and reviewing database schema. Experience with schema management for unstructured data, structured data, relational, star schema.
• Experience with designing and managing semantic data elements (metadata, configuration, master data). Come up with automated pipelines to keep them up-to-date from upstream sources.
• Build and optimize data ingestion, transformation, and storage pipelines for structured and unstructured clinical data. Guide teams that are doing it and ensure support for incremental data processing.
• Ensure data quality, lineage is embedded in all solutions.
• Lead code reviews, proof-of-concepts, and performance tuning for large-scale data systems.
• Collaborate with data governance teams to ensure adherence to UHG and healthcare data standards, lineage, certification, Data use rights, and data privacy.
• Contribute to the maturity of data governance domains and participate in governance councils and working groups.
• Design, Build and monitor MLOps pipelines, model inference and robust piplelines for running AI operations on data.

Secondary Responsibilities

• Mentor data engineers and analysts, fostering a culture of technical excellence and continuous learning.
• Collaborate with product managers, data scientists, and business stakeholders to translate requirements into data solutions.
• Influence architectural decisions across teams and contribute to enterprise-wide data strategy.

• Stay current with emerging technologies in cloud, big data, and AI/ML, and evaluate their applicability to healthcare data.
• Promote the use of generative AI tools (e.g., GitHub Copilot) to enhance development productivity and innovation.
• Drive adoption of DevOps and DataOps practices, including CI/CD, IaC, and automated testing for data pipelines.

Required Skills & Qualifications

• Technical skills
• Ideally 11+ years of experience in data architecture, data engineering, or related roles, with a focus on healthcare or clinical data preferred.
• Proven track record of designing and delivering large-scale data solutions in cloud environments.
• Cloud Platforms: Strong experience with Azure (preferred), AWS, or GCP.
• Big Data Technologies: Proficient in Apache Spark, Databricks, Delta Lake, and distributed data processing.
• Data Engineering: Expertise in building ETL/ELT pipelines, data lakes, and real-time streaming architectures using python, scala or other comparable technologies.
• Data Modelling: Deep understanding of dimensional modeling, canonical models, and healthcare data standards (e.g., HL7, FHIR).
• Programming: Proficiency in Python, SQL, and optionally Scala or Java.
• DevOps/DataOps: Familiarity with CI/CD, IaC (Terraform, ARM)

Soft Skills

• Strong leadership, communication, and stakeholder management skills.
• Ability to mentor and influence across teams and levels.
• Strategic thinker with a passion for data-driven innovation.
• Ability to get into details whenever required and spend time in understanding and solving problems.

Preferred Skills

• Experience with healthcare data interoperability standards (FHIR, HL7, CCD).
• Familiarity with MLOps and integrating data pipelines with ML workflows.
• Contributions to open-source projects or publications in data architecture or healthcare analytics.