Technical Manager - Databricks

This is for Data Management and Delivery. 

Executive Job Description:  Data Management & Delivery 

This is a Strategic delivery role responsible for governing and optimizing the end-to-end Azure-based Data Lakehouse environment, leveraging Databricks as the core processing engine. The focus is to ensure all data initiatives support Airline Industry operations (Flight Ops, IROPs, Revenue Management,etc).

The primary accountability is to shift the organization from operational focus to strategic governance, risk mitigation, and driving data product adoption through engineering excellence. The incumbent holds mandatory experience in pre-sales, solutioning, and technical proposal defense to drive new business growth.

Key Roles & Accountabilities

1. Data Governance, Risk & Business Enablement

This pillar focuses on executive alignment, compliance, and driving the strategic use of data assets.

Act as the executive representative for the Data Governance Council. Ensure platform compliance with regulatory standards and oversee all internal and external data audits. Drive security policies (e.g., PNR data masking) across all layers.

Champion the Data Mesh philosophy and lead the adoption of the Semantic Layer as the official consumption interface. Empower domain teams to create and certify data products from the Gold layer.

Mandate includes leading technical solutioning, cost estimation, and presenting complex data architectures during pre-sales cycles and client demos to secure new engagements.

Lead the Delivery organization, championing the adoption of reusable components and technical standards to enhance development speed and consistency.

2. Platform Resilience & Engineering Excellence (Build/Maintain)

This pillar focuses on engineering quality, scalability, and operational financial health (FinOps).

Own the strategy for designing resilient, back-pressure-free pipelines and infrastructure capable of handling the high-volume, bursty nature of airline data (IROPs) without service degradation.

Own the DQ strategy across the platform. Oversee the building of automated DQ frameworks (e.g., using DLT/Great Expectations) to enforce quality gates between Medallion layers (Bronze, Silver, Gold).

Lead the strategic implementation of an observability framework focused on cost-efficiency (FinOps). Implement controls to monitor, attribute, and optimize Azure consumption (e.g., Databricks DBU optimization, rightsizing clusters).

3. Data Mining, Consumption, & AI Innovation

This pillar focuses on preparing data for advanced use cases and maximizing value for end-users.

Define the technical roadmap for integrating AI in Data Engineering (AIDE), including predictive scheduling and anomaly detection, to automate platform maintenance.

Ensure the Silver and Gold layers are optimized for use by Data Science teams, preparing features and managing large datasets for advanced analytics (e.g., predictive maintenance, dynamic pricing models).

Ensure optimal performance of the consumption layer (e.g., Power BI integration). Enforce Zero-Copy Data Sharing (Delta Sharing) from the Gold layer to eliminate duplication.

Ensure the Semantic Layer provides the foundational, standardized metrics necessary to enable self-service advanced analytics and Generative AI initiatives.

Required Skills and Experience (15+ Years)

15+ years in Data Engineering/Architecture, with 5+ years managing global DataOps/Platform teams.

Mandatory experience in the Airline or Transportation industry, specifically operational data (Flight Ops, IROPs, PNR, Baggage Tracking).

Expert-level skills in the Azure Cloud Platform (ADLS Gen2, Event Hubs) and Databricks (Spark optimization, DBU management, Delta Lake).

Expert proficiency in Resilience Engineering (back pressure prevention, circuit breakers), Cost-Optimized Observability (FinOps), and building reusable components.

Mandatory expertise in solution architecture, technical proposal writing, and leading executive-level pre-sales/client presentations.

Proven executive-level ability to define and execute complex multi-year data strategy roadmaps, coupled with strong executive communication and stakeholder management skills.