AWS Databricks Architect

Role & responsibilities

Role Overview

Databricks

Location:

Key Responsibilities

Architecture & Strategy

• Define the enterprise data architecture vision and standards for Lakehouse and cloud-native platforms.
• Develop reference patterns for ingestion, transformation, modeling, and governance using Databricks, and AWS.
• Drive adoption of modern frameworks such as DBT for transformation and Airflow for orchestration.
• Enable privacy-preserving analytics through Clean Room solutions and architect reusable data products.
• Ensure designs support scalability, cost optimization, and future-proofing for advanced analytics and AI.
• Drive adoption of Medallion architecture and advanced Lakehouse capabilities (Photon, Delta Live Tables, AI/BI).
• Drive API-based integrations for data services and interoperability.
• Evaluate new AWS and Databricks features, cost models, runtimes, governance controls, and roadmap impact.

Technical Design Leadership

• Provide architectural guidance to engineering teams for building pipelines and APIs.
• Establish reusable frameworks for ETL/ELT, streaming, and batch processing.
• Integrate Power BI for analytics and reporting, ensuring seamless connectivity with Lakehouse architecture.
• Implement telemetry-driven observability using Splunk/Datadog and enforce DevOps best practices for CI/CD.
• Guide teams on modular designs leveraging Microservices, containerization (Docker/Kubernetes), and infrastructure automation (Terraform) as good-to-have capabilities.
• Review design specifications, data models, pipeline patterns, and solution architectures across squads.
• Ensure cloud-native designs aligned with AWS platform best practices.

Governance, Security & Optimization

• Define standards for data governance, lineage, metadata, and cataloging (e.g., Atlan, Unity Catalog).
• Implement security policies, IAM controls, and compliance frameworks across cloud environments.
• Optimize performance and cost through cluster strategy, workload tuning, and automation.
• Establish observability and telemetry frameworks for operational health and proactive issue resolution.

Cross-Functional Collaboration

• Partner with enterprise architects, product owners, and business stakeholders to validate architectural fit for new and existing use cases.
• Support migration strategies from legacy platforms to modern architectures.