Databricks Implementation Lead

• Experience needed:

   8-12 years

• Type:

   Full-Time

• Mode:

   WFO

• Shift:

  IST

• Location:

   Hyderabad, India

• Notice Period:

   Immediate to 15 days

About the Role

We are seeking an experienced

data architecture

Key Responsibilities

1. Project Leadership

• Own the complete 

  Databricks implementation lifecycle

   from requirement gathering, architecture design, and environment setup to production deployment and handover.
• Act as the primary 

  technical liaison

   between business stakeholders, data engineering teams, and cloud platform teams.
• Ensure alignment with project goals, timelines, and compliance standards (

  data governance, security, cost optimization

  ).

2. Architecture & Design

• Design scalable and secure 

  data pipelines

   leveraging 

  Databricks, Delta Lake

  , and 

  cloud-native services (Azure, AWS, or GCP)

  .
• Define the 

  data ingestion, transformation, and consumption architecture

  , integrating multiple data sources (APIs, databases, files, streaming).
• Establish 

  CI/CD pipelines

   for Databricks notebooks and jobs, enabling automation and version control (using Git / Azure DevOps).

3. Technical Implementation

• Lead and mentor a team of 

  data engineers

   to develop 

  ETL/ELT pipelines

   using 

  PySpark, SQL, and Databricks workflows

  .
• Configure clusters, jobs, and 

  Lakehouse storage

   following best practices.
• Optimize 

  Databricks compute and query performance

   for cost and efficiency.
• Implement 

  data quality frameworks, monitoring dashboards, and error-handling logic

  .

4. Governance & Operations

• Set up 

  access control, role-based security, and data lineage tracking

   in 

  Unity Catalog

  .
• Collaborate with the 

  CloudOps team

   for environment setup, monitoring, and cost governance.
• Drive 

  documentation, training, and knowledge transfer

   to ensure sustainability and handover readiness.

Required Skills & Experience

• 8+ years of experience in 

  data engineering or big data projects

  .
• 3+ years of 

  hands-on Databricks experience

   (SQL, PySpark, Delta Lake, MLflow).
• Strong experience with 

  ETL/ELT pipeline design

  , 

  data modeling

  , and 

  data warehousing

  .
• Experience with 

  Azure Data Factory, AWS Glue

  , or similar orchestration tools.
• Proficiency in 

  Python, SQL

  , and 

  Spark optimization techniques

  .
• Experience implementing 

  DevOps practices (CI/CD, Git, Databricks Repos)

  .
• Familiarity with 

  data governance frameworks, Unity Catalog, and RBAC

  .
• Proven track record in 

  leading multi-member implementation teams

  .

Preferred Qualifications

• Databricks Certified Data Engineer Professional

   or 

  Solution Architect certification

  .
• Experience with 

  AI/ML workflows

   and model management using 

  MLflow

  .
• Exposure to 

  Power BI, Tableau, or Looker

   for downstream visualization.
• Strong communication skills and experience in 

  client-facing roles

  .

Soft Skills

• Excellent 

  problem-solving and analytical skills

  .
• Strong 

  leadership and mentoring

   capabilities.
• Ability to work in a 

  fast-paced, cross-functional environment

  .
• Clear 

  documentation and presentation

   skills.