DXC - Lead Databricks Engineer - Python/SQL

About The Role

We are seeking a highly skilled Lead Databricks Engineer to join our team at DXC Technology, where you will play a pivotal role in building and scaling enterprise-grade data solutions on the Azure ecosystem. Youll lead the development of robust data pipelines, analytics frameworks, and cloud-native solutions using cutting-edge tools like Azure Databricks (ADB), Azure Data Factory (ADF), Delta Lake, and PySpark.

Responsibilities

This role is ideal for professionals who thrive in fast-paced, collaborative environments and are passionate about data engineering, cloud transformation, and data-driven business Responsibilities :

• Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Databricks, PySpark, and ADF.
• Write, optimize, and manage complex SQL queries and transformations across structured and unstructured data sources.
• Implement data lake solutions using Delta Lake, Unity Catalog, and Azure Synapse Analytics.
• Apply data profiling, cataloging, and lineage techniques to ensure data quality and governance compliance.
• Enable secure and governed access to data assets using Azures data governance features and role-based access control (RBAC).
• Serve as technical lead on large-scale data engineering initiatives, guiding junior developers and ensuring architectural best practices.
• Lead Proof of Concepts (PoCs), prototype development, and pilot solution implementations to validate technical feasibility.
• Work closely with solution architects, business analysts, DevOps teams, and client stakeholders to translate requirements into data-driven outcomes.
• Take part in business process mapping and contribute to data architecture decisions aligned with business needs.
• Utilize Azure DevOps for CI/CD, automation of data pipelines, and environment management.
• Establish and enforce coding standards, data modeling conventions, and deployment workflows.
• Monitor and optimize the performance of data pipelines, ETL/ELT processes, and data storage systems.
• Troubleshoot and resolve performance bottlenecks and errors in production Qualifications :
• 5+ years of hands-on experience as a Data Engineer in Azure cloud environments.
• Azure Databricks (ADB)
• Azure Data Factory (ADF)
• Delta Lake / Delta Tables
• Synapse Analytics
• Unity Catalog
• Strong programming proficiency in Python, PySpark, and Spark SQL.
• Deep understanding of SQL with the ability to write complex queries, joins, window functions, etc.
• Experience with Azure DevOps for pipeline automation, code repositories (Git), and CI/CD integration.
• Demonstrated ability to conduct data profiling, data cataloging, and data lineage tracking.
• Strong knowledge of data governance, security, and compliance within cloud data environments.
• Excellent analytical thinking, problem-solving, and communication skills.
  

(ref:hirist.tech)