Fabric Data Engineer (Remote)

Microsoft Fabric for a US health insurer

Job Title

Location

Working Hours

Experience Required

Education

Preferred certifications

Who are you

• SQL & CDC Pro:

  Strong SQL Server/T-SQL; hands-on CDC or replication patterns for initial snapshot + incremental syncs, including delete handling.

• Fabric Mirroring Practitioner:

  You've set up and tuned Fabric Mirroring to land data into OneLake/Lakehouse; comfortable with OneLake shortcuts and workspace/domain organisation.

• Schema-Drift Aware:

  You detect, evolve, and communicate schema changes safely (contracts, tests, alerts), minimising downstream breakage.

• High-Volume Ingestion Mindset:

  You design for throughput, resiliency, and backpressureretries, idempotency, partitioning, and efficient file sizing.

• Python/Scala/Spark Capable:

  You can build notebooks/ingestion frameworks for advanced scenarios and data quality checks.

• Operationally Excellent:

  You add observability (logging/metrics/alerts), document runbooks, and partner well with platform, security, and analytics teams.

• Data Security Conscious:

  You respect PII/PHI, apply least privilege, and align with RLS/CLS patterns and governance guardrails.

What you will be doing

• Stand Up Mirroring:

  Configure Fabric Mirroring from SQL Server (and other relational sources) into OneLake; tune schedules, snapshots, retention, and throughput.

• Land to Bronze Cleanly:

  Define Lakehouse folder structures, naming/tagging conventions, and partitioning for fast, organised Bronze ingestion.

• Handle Change at Scale:

  Implement CDCincluding soft/hard deletes, late-arriving data, and backfillsusing reliable watermarking and reconciliation checks.

• Design Resilient Pipelines:

  Build ingestion with Fabric Data Factory and/or notebooks; add retries, dead-lettering, and circuit-breaker patterns for fault tolerance.

• Manage Schema Drift:

  Automate drift detection and schema evolution; publish change notes and guardrails so downstream consumers aren't surprised.

• Performance & Cost Tuning:

  Optimise batch sizes, file counts, partitions, parallelism, and capacity usage to balance speed, reliability, and spend.

• Observability & Quality:

  Instrument lineage, logs, metrics, and DQ tests (nulls, ranges, uniqueness); set up alerts and simple SLOs for ingestion health.

• Collaboration & Documentation:

  Partner with the Fabric Platform Architect on domains, security, and workspaces; document pipelines, SLAs, and runbooks.

Must-have skills

• SQL Server, T-SQL; CDC/replication fundamentals
• Microsoft Fabric Mirroring; OneLake/Lakehouse; OneLake shortcuts
• Schema drift detection/management and data contracts
• Familiarity with large, complex relational databases
• Python/Scala/Spark for ingestion and validation
• Git-based workflow; basic CI/CD (Fabric deployment pipelines or Azure DevOps)

Benefits

• 5 day work week
• 100% remote setup with flexible work culture and international exposure
• Opportunity to work on mission-critical healthcare projects impacting providers and patients globally