Senior Data Engineer

As a Senior Data Engineer, you will build the foundational pipelines that bring together multi-cloud billing and operational data. You will make data dependable, traceable, and easy for downstream teams to consume by supporting cost visibility, budget monitoring, anomaly detection, shared-cost allocation, and cost-to-serve reporting. This is an execution-oriented engineering role with end-to-end ownership of pipeline components, including source onboarding, normalization, transformation, validation, monitoring, incident response, and documentation.

This is an individual contributor role reporting to the Senior Director, Engineering.

Responsibility

• Build batch and/or streaming pipelines for Azure and GCP billing data, with an architecture that can extend to additional cloud providers

• Normalize provider-specific billing and usage data into a consistent, FOCUS-aligned model

• Ingest and join resource inventory, tagging, ownership, product, service, and environment metadata

• Implement reconciliation checks between source billing systems, the normalized warehouse, and downstream reporting datasets

• Establish data contracts, schemas, lineage, freshness checks, completeness checks, and failure-handling patterns

• Create reusable transformations that support executive dashboards, product-level cost views, burn-rate reporting, and anomaly detection

• Build operational visibility for pipeline health, including freshness, latency, volume, schema drift, and failed-record monitoring

• Partner with platform, FinOps, Finance, and service-owning teams to resolve data-quality gaps and improve attribution coverage

• Document pipeline behavior, ownership, runbooks, recovery procedures, and known data limitations

• Improve automation, test coverage, deployment safety, and cost efficiency of the data platform

• Build or integrate event-driven ingestion patterns so new Cost Management exports trigger processing without manual refreshes

• Support Microsoft Fabric, Data Lake, Data Factory, and KQL-based consumption patterns where they fit the target architecture

• Treat private endpoints, private DNS, managed identities, Entra ID/RBAC, and disabled public access as part of the data-platform design—not afterthoughts

• Participate in on-call rotations and contribute to improving system reliability and incident response

Basic

• 5+ years of professional experience designing, implementing, and operating production data pipelines

• Strong SQL and Python or an equivalent data-engineering language

• Experience with cloud billing, usage, infrastructure, telemetry, or other operational datasets

• Experience with dimensional modeling, data warehouses/lakes, ETL/ELT, and data-quality testing

• Practical experience with pipeline orchestration, monitoring, alerting, and incident response

Preferred

• Multi-cloud billing data or FinOps data engineering

• FOCUS, cloud-cost normalization, resource tagging, or showback/chargeback systems

• Azure, GCP, BigQuery, Microsoft Fabric, Azure Data Factory, Data Lake Storage, KQL, or comparable warehouse technologies

• Event Grid or equivalent event-driven ingestion, managed identities, Entra ID/RBAC, private endpoints, private DNS, and network-isolated data services

• Data lineage, schema evolution, reconciliation, anomaly detection, deployment verification, or finance-system integration

• Ability to work across engineering and business stakeholders to clarify data definitions and ownership