Senior Technical Lead

Pune

Posted:2 days ago

Job Description

Job Summary

Data Platform Engineer with a strong focus on Databricks and Azure to design, build, and operate a scalable and secure data platform. You will work at the intersection of infrastructure, data engineering, and cloud automation to establish an enterprise platform for analytics and AI workloads

Key Responsibilities
  • Automate the provisioning of Databricks and Azure platform resources using Terraform, adhering to Infrastructure-as-Code (IaC) best practices
  • Design and develop reusable, modular Terraform components to enable scalable and maintainable deployments within the Databricks platform
  • Implement and maintain CI/CD pipelines using Azure DevOps for automated infrastructure and platform deployments
  • Provision new infrastructure using customized Databricks Terraform modules to support requests for new catalogs, job policies, and clusters in a standardized, automated, and governed manner
  • Enable self-service infrastructure provisioning by integrating an existing ServiceNow interface with automated workflows that generate pull requests for new Databricks resources, ensuring standardized and governed deployments
  • Troubleshoot and resolve platform-related issues to ensure high availability and uninterrupted service, including proactive monitoring and management of Azure quota limits, cluster performance, resource utilization, and platform constraints
  • Drive FinOps integration by onboarding Databricks service cost data into the existing FinOps (Umbrella) system, including configuring required data formats, enabling cost monitoring and reporting, and ensuring transparency and optimized resource usage across Azure and Databricks
  • Stay up to date with industry best practices and emerging technologies, especially Terraform (Azure Provider, Databricks Provider)
Skill Requirements
  • Degree in Computer Science, Information Technology, or a related field.
  • Proven experience as a Platform Engineer/Cloud Engineer or similar role.
  • Strong understanding of cloud platforms (preferably Microsoft Azure) and their services.
  • Good knowledge of infrastructure as code (IaC) using Terraform. Familiarity with Databricks Provider is an advantage.
  • Experience with Azure Data Lake Gen2 setup using Terraform
  • Knowledge in data management (e,g. archiving, backup) and the setup of interfaces between systems along with authentication mechanisms
  • Excellent problem-solving skills and the ability to work in a fast-paced environment.
  • Strong communication and collaboration skills.
  • Excellent English skills. German skills are a bonus.
  • Certfications in Microsoft Azure or Databricks are welcome.
Must Have Skill
Azure Databricks,Azure DevOps,Terraform
Good to Have Skill
Other Requirements

Skills Required

Azure Data Engineering Automation Analytics Terraform Ci/cd Pipelines Azure Devops Servicenow High Availability Resource Utilization Onboarding Emerging Technologies

Learn the skills this job asks for

See your skill gap for Software Developer →

Free courses, interview questions and practice for each skill.

Mock Interview

Practice Video Interview with JobPe AI

Start Azure DevOps Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
HCLTech logo
HCLTech

Information Technology Services

New Delhi

RecommendedJobs for You

infinite Computer Solution logo

Senior Technical Lead

infinite Computer Solution

India

infinite Computer Solution logo

Senior Technical Lead

infinite Computer Solution

Pune