Databricks Architect

Databricks Architect

Location:

About the Role:

Databricks Architect

Architecture & Design

• Design and implement end-to-end lakehouse solutions using Databricks on Azure/AWS.
• Architect modular, reusable components for Delta Lake, MLflow, Unity Catalog, and Databricks Workflows.
• Develop multi-layer data pipelines (bronze-silver-gold) with automated data quality checks and lineage tracking.

Development & Delivery

• Build job clusters, notebooks, and Delta Live pipelines with CI/CD automation.
• Collaborate with Product and Engineering teams to ensure product–platform alignment.
• Define and enforce best practices for performance tuning, FinOps, and cluster cost optimization.
• Support data science and MLOps integration — serving models via Databricks Model Serving or external APIs.

Innovation & Enablement

• Lead the Databricks Intelligence Platform adoption and implementations — including AI/BI capabilities, GenAI agents, and natural-language queries.
• Contribute to internal accelerators, frameworks, and demo environments.
• Mentor engineers and solution teams on Databricks ecosystem and coding standards.

Required Skills & Experience

• 12+ years in Data Engineering / Architecture with 5+ years in Databricks.
• Proven expertise with:
• Databricks Intelligence Platform (Lakehouse, Unity Catalog, Delta Live Tables, AI/BI)
• Delta Lake design, schema evolution, and ACID optimization
• PySpark, SQL, and DBT for ETL and transformation pipelines
• Azure Data Factory, AWS Glue, or Airflow for orchestration
• MLflow and model lifecycle management
• Hands-on experience integrating Databricks with Power BI, Fabric, or external AI systems.
• Strong grounding in data governance, quality frameworks, and observability tools.
• Certification preferred: Databricks Certified Data Engineer Professional or Databricks Certified Architect.