Data Engineer

Key Responsibilities

• Design, develop, test, and deploy scalable pipelines using Databricks, PySpark, Spark SQL, and Delta Lake, spanning Bronze/Silver/Gold (medallion) layers for HR data integration and analytics.

• Build data models – logical, physical, dimensional, and integration – and maintain source-to-target mappings for workforce, organization, employee, position, job, and related HR domains.

• Migrate legacy GDC integrations by re-engineering Informatica, SQL, and stored-procedure logic into modern Databricks patterns, including inbound Oracle HCM feeds and outbound HR data files.

• Validate output parity through reconciliation, parallel runs, and source-to-target checks, and build automated data quality, exception handling, and audit controls into pipelines.

• Apply enterprise governance, security, privacy, and role-based access expectations to sensitive HR data, capturing lineage and metadata where applicable.

• Produce production-ready documentation – technical designs, runbooks, data dictionaries, and support handoffs – for maintainable, observable, cost-aware solutions.

• Collaborate within a distributed delivery team, participate in agile ceremonies, mentor junior engineers, and communicate progress, blockers, and risks clearly.

Required Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field, with 7+ years of data engineering, ETL, or data platform experience.

• Azure Databricks & Apache Spark: 3+ years hands-on building Delta Lake pipelines with PySpark, Spark SQL, and SQL across a medallion architecture.

• Data modeling & integration: Strong logical/physical/dimensional modeling plus source-to-target mapping, reconciliation, and production cutover support.

• AI-assisted engineering: Proficiency using AI/Copilot tooling to accelerate development, code conversion, and documentation.

• Practical, delivery-focused work style, comfortable turning ambiguous legacy logic into clean target-state designs within a highly governed environment.

• Strong written and verbal communication, with the ability to overlap U.S. Eastern Time for key meetings, design reviews, and issue resolution.

Preferred Qualifications

• Suggested 3+ years modernizing legacy ETL (Informatica-to-Databricks) or building metadata-driven, configuration-driven data frameworks.

• Azure ecosystem: Data Lake Storage, Data Factory, DevOps, Azure SQL/Synapse, Unity Catalog, and Git-based CI/CD.

• HR domain data: Oracle HCM Cloud extracts/PVOs/HDL, workforce analytics, and consumption via Power BI, Crunchr, or similar.

• Experience supporting regulated, sensitive employee data in insurance, financial services, or HR technology.

• Detail-oriented, collaborative, and comfortable working independently against offshore delivery expectations while staying aligned to MetLife leads.