Senior Data Engineer

We are seeking a Senior Data Engineer to support our strategic data initiatives. As we implement a Medallion Architecture (Bronze/Silver/Gold layers) and replatform several core transactional systems, this role will be critical in ensuring data quality, governance, and scalability. You will focus on operationalizing Silver and Gold data layers, adapting pipelines to new data sources, and ensuring downstream analytics, dashboards, and reporting remain reliable during this transition.

Key Responsibilities:

Silver/Gold Layer Development

• Translate raw data from sources (Zuora, CommerceTools, APIs, legacy databases) into curated Silver and Gold tables.
• Apply business logic, standardization, and data quality checks.
• Partner with analysts and business stakeholders to validate metric definitions.

Replatforming Support

• Audit existing pipelines for dependencies on legacy data structures.
• Update models and jobs to consume data from new transactional systems.
• Ensure lineage tracking and compatibility with downstream reporting and analytics.

Collaboration & Documentation

• Work closely with engineers, analysts, and business partners to document requirements, transformations, and validations.
• Contribute to metadata tagging and data cataloging (Unity Catalog).
• Deliver detailed documentation including data dictionaries, constraints (PK, FK), and pipeline logic.

Requirements

• Minimum 7+ years of relevant experience
• Strong data modeling skills (Kimball methodology) with hands-on Databricks experience.
• Proficiency in SQL and Python
• Expertise with Databricks features such as workflows, DLTs, Delta tables, and Unity Catalog.
• Familiarity with Medallion Architecture (Bronze/Silver/Gold layers).
• Experience with version control systems (GitHub).
• Solid understanding of data governance, data quality frameworks, and observability practices.
• Ability to work effectively across analytics, product, and engineering teams.

Preferred skills:

• Experience with replatforming large transactional systems.
• Exposure to subscription and commerce data domains