Senior Databricks Engineer

We’re Hiring:

Location

Work Mode

Availability

Note

 🧭 L2 interviews will be conducted in person — apply only if you are open to face-to-face interviews.

🌟 About Newscape Consulting-

Newscape is a fast-growing digital services company specializing in healthcare technology and data engineering. We help healthcare organizations modernize legacy systems, integrate cloud-native platforms, and drive digital transformation through data.

 If you’re passionate about solving complex data challenges and contributing to meaningful innovation in healthcare, this is the place for you.

💼 Position Summary -

This role requires a seasoned Databricks professional with strong technical depth and leadership capability. You will design, architect, and optimize data pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native services (Azure, AWS, or GCP). As a Senior or Lead engineer, you will guide best practices, mentor junior engineers, and collaborate closely with data architects, data scientists, and business stakeholders.

Key Responsibilities

• Architect, design, and maintain large-scale data pipelines using Databricks, PySpark, Spark SQL, and optionally Scala.
• Lead the adoption and implementation of Databricks components including Workspace, Jobs, DLT (Delta Live Tables), Repos, and Unity Catalog.
• Build and optimize Delta Lake solutions aligned with Lakehouse and Medallion architecture standards.
• Partner with architects, engineering teams, and business stakeholders to translate complex business requirements into robust technical solutions.
• Establish coding standards, best practices, and reusable components for Databricks-based development.
• Drive CI/CD automation for Databricks deployments using Azure DevOps, GitHub Actions, or equivalent tools.
• Ensure end-to-end data governance, data quality, metadata management, and lineage tracking leveraging Unity Catalog or Azure Purview.
• Utilize orchestration tools such as Apache Airflow, Azure Data Factory, or Databricks Workflows for scheduling and monitoring.
• Conduct performance tuning for Spark jobs, cluster optimization, and cost-efficient compute strategies.
• Guide data modeling, SQL transformations, and data warehousing implementation using industry-standard best practices.
• Mentor and provide technical leadership to junior and mid-level data engineers.
• Participate in architecture reviews, solution design discussions, and roadmap planning.

Required Skills and Qualifications

• Strong hands-on experience in data engineering with specific and extensive expertise in Databricks.
• Deep proficiency with Databricks Workspace, Jobs, DLT, Repos, Unity Catalog, and Lakehouse architecture.
• Advanced programming experience with PySpark and Spark SQL; Scala is an added advantage.
• Strong experience with at least one major cloud platform: Azure, AWS, or GCP.
• Proven expertise with Delta Lake, Lakehouse, and Medallion architecture patterns.
• Hands-on experience implementing CI/CD pipelines for Databricks using DevOps tooling.
• Familiarity with orchestration tools such as Airflow, ADF, or Databricks Workflows.
• Strong understanding of data governance, security frameworks, and enterprise data management.
• Experience integrating Databricks datasets with Power BI, including creating optimized data models, implementing Direct Lake or DirectQuery connections, and enabling performant, enterprise-grade reporting solutions.
• Experience leading technical teams or acting as a senior SME in critical data projects.
• Excellent analytical abilities, communication skills, and stakeholder management capability.
• Prior experience in healthcare data environments is a plus (not mandatory).

Work Model

• This is a full-time, onsite role based in Baner, Pune.
• 5 days onsite per week is mandatory.

Ready to make an impact?

Apply now or DM us if you’re a Pune-based data enthusiast

📩 How to Apply:

 Send your resume to 👉 swapnil.supe@newscapeconsulting.com

 📌 Subject Line: Application – Sr. Databricks Engineer

 📞 +91 8233829595