Databricks Architect

Role & responsibilities

Role Overview

The Databricks Architect is responsible for defining the enterprise-wide Lakehouse architecture, setting technical direction, and overseeing the design of scalable, secure, and high-performance data solutions on the Databricks platform. This role provides architectural governance, ensures best-practice adoption across teams, and guides Databricks Tech Leads in delivering future-proof data engineering, analytics, and AI solutions.

Location:

Key Responsibilities

Architecture & Strategy

• Define Lakehouse architecture vision, standards, and reference patterns using Delta Lake, Unity Catalog, Databricks SQL, and ML/AI runtimes.
• Lead architecture blueprinting for ingestion, transformation, modelling, governance, and AI enablement.
• Drive adoption of Medallion architecture and advanced Lakehouse capabilities (Photon, Delta Live Tables, AI/BI).
• Evaluate new Databricks features, cost models, runtimes, governance controls, and roadmap impact.

Technical Design Leadership

• Provide architectural guidance to Databricks Tech Leads and development teams.
• Review design specifications, data models, pipeline patterns, and solution architectures across squads.
• Establish reusable frameworks, templates, and engineering standards for ETL/ELT, streaming, batch, and ML pipelines.
• Ensure cloud-native designs aligned with AWS/Azure/GCP platform best practices.

Governance, Security & Optimization

• Define and govern standards for Unity Catalog, lineage, metadata, glossary, security policies, and governance operating models.
• Ensure performance tuning, cluster strategy, CI/CD policies, and cost optimization across environments.
• Guide teams on data quality frameworks, observability, telemetry, and operational monitoring.

Cross-Functional Collaboration

• Partner with enterprise architects, data product owners, analytics teams, and business stakeholders.
• Validate architectural fit for new use cases, solutions, and data products.
• Support pre-sales, estimations, and sizing for new programs and migrations (Hadoop Databricks, DWH Lakehouse).

AI/LLM Enablement (Advantage)

• Design frameworks for agentic workflows, GenAI-based automation, code generation, data quality validation, and metadata enrichment.
• Enable Databricks Model Serving, Vector Search, and AI/BI integration where applicable.

Required Skills & Experience

• 1218+ years in data engineering/architecture with 4–6+ years in Databricks.
• Deep expertise in:
• PySpark, Spark SQL, Delta Lake, Unity Catalog
• Databricks Workflows, Delta Live Tables (DLT), SQL Warehouses, Photon
• Streaming, batch, and real-time architectures
• Strong background in cloud architecture (AWS/Azure/GCP), IAM, networking, and security.
• Strong understanding of data governance, lineage, metadata, and cataloguing platforms.
• Hands-on experience designing enterprise-scale Lakehouse platforms.
• Experience guiding teams of Tech Leads and Engineers.

Preferred candidate profile

• Databricks Certified Data Engineer Professional or Databricks Architect certification.
• Experience with Databricks AI/BI, Lakehouse Federation, vector search, and Model Serving.
• Experience leading migrations from Hadoop, on-prem data warehouses, or legacy ETL platforms.
• Knowledge of LLMs, agentic frameworks, or GenAI automation (strong advantage).