Lead Databricks Engineer / Architect

Job Description:

Lead Databricks Engineer / Architect

Job Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / Mumbai

Role Overview
We are seeking an experienced Lead Data Engineer / Architect with over 10 years of industry experience (5+ years in Databricks and modern data platforms) to design, implement, and lead enterprise-scale data solutions. The ideal candidate is hands-on with Databricks ecosystem, PySpark (UDFs, Pandas UDFs), dbt, and Airflow, and is capable of driving architecture decisions, leading engineering teams, and ensuring best practices in data engineering. Key Responsibilities

• Lead the design and implementation of data lakehouse architectures using Databricks, Delta Lake, and Delta Live Tables.
• Define best practices for workspace setup, cluster management, repos, and job orchestration.
• Partner with stakeholders to translate business needs into scalable data engineering solutions.
• Data Engineering & Development
• Develop and optimize ETL/ELT pipelines using PySpark, dbt, and Airflow.
• Implement advanced PySpark transformations leveraging UDFs and Pandas UDFs for complex use cases.
• Ensure data quality, lineage, and observability across all pipelines.

Required Skills & Qualifications

• 10+ years of total IT experience, with at least 5 years of relevant data engineering experience on Databricks.
• Strong expertise in:
• Databricks (Workspace, Clusters, Jobs, Repos, Delta Live Tables)
• PySpark (UDFs & Pandas UDFs)
• dbt for data transformations
• Apache Airflow for orchestration
• Solid understanding of cloud data platforms (Azure/AWS/GCP).
• Hands-on with CI/CD, version control (Git), and DevOps practices for data engineering.
• Databricks Certified Data Engineer Associate (mandatory).
• Proven ability to lead and mentor teams, with excellent communication and stakeholder management skills.

At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We re committed to fostering an inclusive environment where everyone can thrive.

Recruitment fraud is a scheme in which fictitious job opportunities are offered to job seekers typically through online services, such as false websites, or through unsolicited emails claiming to be from the company. These emails may request recipients to provide personal information or to make payments as part of their illegitimate recruiting process. DXC does not make offers of employment via social media networks and DXC never asks for any money or payments from applicants at any point in the recruitment process, nor ask a job seeker to purchase IT or other equipment on our behalf. More information on employment scams is available here .