Databricks Data Engineer

About the Role:

We are seeking an experienced Databricks Data Engineer with 5+ years of experience to design, build, and optimize scalable, high-performance data pipelines and analytics solutions using the Databricks platform. The ideal candidate will play a key role in architecting data platforms and mentoring junior engineers.

Key Responsibilities

• Design, develop, and optimize large-scale ETL/ELT pipelines using Databricks, Spark, and Python .
• Architect and manage Delta Lake tables for high-performance analytics.
• Ingest and process structured and unstructured data from multiple sources.
• Optimize Spark workloads for performance, scalability, and cost efficiency .
• Implement robust data quality, monitoring, logging, and alerting frameworks.
• Lead integration with cloud services (Azure/AWS/GCP) and data orchestration tools.
• Drive CI/CD automation for data pipelines and notebook deployments.
• Enforce data security, governance, and compliance standards.
• Collaborate with architects, data scientists, analysts, and business stakeholders.
• Mentor junior engineers and drive engineering best practices.

Ideal Candidate Profile

• 5+ years of hands-on experience in Data Engineering.
• Strong expertise in Databricks Apache Spark.
• Advanced proficiency in Python PySpark.
• Strong command of SQL performance tuning.
• Hands-on experience with Delta Lake Parquet.
• Solid experience with cloud platforms (Azure/AWS/GCP).
• Strong knowledge of data warehousing concepts (Star Schema, SCD).
• Experience with ETL/ELT orchestration tools (ADF, Airflow, etc.).
• Exposure to CI/CD automation and Git-based version control.
• Strong understanding of data security, governance, and access control.
• Excellent communication, leadership, and problem-solving skills.
• Experience working in Agile/Scrum delivery models.

Good to have

• Databricks Certified Data Engineer (Associate/Professional)
• Experience with real-time/streaming data (Kafka, Event Hubs)
• Exposure to ML workflows in Databricks

Why Join Exponentia.ai

• Innovate with Purpose: Opportunity to create pioneering AI solutions in partnership with leading cloud and data platforms
• Shape the Practice: Build a marquee capability from the ground up with full ownership
• Work with the Best: Collaborate with top-tier talent and learn from industry leaders in AI
• Global Exposure: Be part of a high-growth firm operating across US, UK, UAE, India, and Singapore
• Continuous Growth: Access to certifications, tech events, and partner-led innovation labs
• Inclusive Culture: A supportive and diverse workplace that values learning, initiative, and ownership

Ready to build the future of AI with us

• Apply now and become a part of a next-gen tech company that s setting benchmarks in enterprise AI solutions.