Sr. Databricks Platform Admin

Job Details

We are hiring a Senior Databricks Platform Administrator to own the Azure Databricks platform behind Associa's Data Lake.

You will run the workspaces, Unity Catalog, identity, networking and infrastructure as code that our data engineers, analysts and data scientists build on. You will be the team's authority on Terraform, Azure and Entra ID authentication for Databricks, accountable for a secure, governed, reliable and cost-controlled platform.

Terraform: own the Terraform codebases for Azure infrastructure and Databricks configuration, including modules, remote state, provider upgrades, drift and imports. Keep every platform change in code and review the team's Terraform pull requests.

Azure: manage VNet-injected workspaces, private endpoints and DNS, ADLS Gen2, Key Vault, managed identities and Azure RBAC.

Identity and authentication: own the Entra ID integration with Databricks: SCIM sync, groups, entitlements, service principals, OAuth and OIDC workload identity federation. Keep access least-privilege.

Unity Catalog: administer the metastore, catalogs, external locations, storage credentials, grants and Delta Sharing as code.

Compute: own cluster policies, SQL warehouses, serverless settings and runtime upgrades, and keep usage and cost under control.

CI/CD: own the Azure DevOps pipelines for Terraform and Databricks Asset Bundles, including service connections, environments and approvals.

Operations: configure audit logging and system tables, act as escalation point for platform incidents and Databricks support, plan for Databricks platform changes, and document the platform.

10+ years in cloud infrastructure or platform engineering, with hands-on Azure Databricks administration.

Expert Terraform: modules, remote state, imports, provider upgrades, and plan/apply through CI/CD with the azurerm, azuread and databricks providers.

Deep Azure knowledge: VNet injection, Private Link and DNS, ADLS Gen2, Key Vault, managed identities and RBAC.

Strong Entra ID and authentication expertise: SCIM, service principals, OAuth, OIDC and workload identity federation.

Unity Catalog and Databricks account and workspace administration.

Azure DevOps YAML pipelines.

Scripting in PowerShell, Bash or Python, and comfort with the Databricks CLI and REST APIs.

Databricks Platform Administrator certification, or Microsoft AZ-104, AZ-305 or AZ-500.

HashiCorp Terraform certification.

Experience with Databricks Asset Bundles, Delta Sharing, Apps or model serving.

Data engineering knowledge (Spark, Delta Lake, Lakeflow jobs and pipelines), enough to support the engineers who build on the platform.