Senior Technical Lead

5 years

Pune

Posted:2 days ago

Job Description

Job Summary

Data Engineer (Databricks) designs and maintains scalable, reliable data pipelines that transform raw data into analytics‑ready datasets for reporting and decision‑making. The role ensures high data quality, security, and performance across cloud‑based ETL workflows, while collaborating closely with analytics and business teams to enable trusted, data‑driven insights.

Key Responsibilities
  • Design, develop, and maintain scalable data pipelines and ETL processes
  • Implement ETL processes to ingest, transform, and load data from various sources
  • Ensure data quality, security, and compliance
  • Optimize and fine-tune data systems for performance and scalability
  • Develop and document data architecture and processes
  • Collaborate with data scientists and analysts to understand data requirements
  • Work closely with cross-functional teams to support data-driven decision-making
  • Monitor and troubleshoot data pipeline issues
  • Troubleshoot and resolve data-related issues and support users
  • Monitor ETL processes, proactively identifying and resolving issues or performance bottlenecks to maintain high system reliability.
  • Review data transformation and ingestion patterns, including CDC, schema evolution, and error handling for robust data operations.
  • Document ETL workflows, data mappings, and process changes to ensure transparency and clear communication across technical and non-technical teams.
  • Stay updated with the latest features and best practices
Skill Requirements
  • Degree in Computer Science, Data Engineering, Information Technology or a related field
  • A minimum of 5 years of professional experience in data engineering or a related role
  • Proven experience as a Data Engineer with a focus on Databricks
  • Strong programming skills in SQL and Python
  • Experience with cloud computing platforms (preferably Microsoft Azure)
  • Ideally Experience with data virtualization tools (Denodo)
  • Strong problem-solving and analytical skills
  • Proficiency in CDC ingestion patterns (watermarks, change tables, log-based ingestion).
  • Advanced error handling & observability
  • Excellent communication and teamwork abilities
  • A passion for working with data and a desire to continuously learn and improve
  • Excellent English skills. German skills are a bonus
  • Microsoft Azure or Databricks certifications are a plus

Skill Level (SFIA):

  • 4 = Enable: Performs diverse complex activities, supports and guides others, delegates tasks when appropriate, works autonomously under general direction, and contributes expertise to deliver team objectives.
Must Have Skill
Agile Methodology,Azure Databricks,PySpark
Good to Have Skill
Other Requirements

Skills Required

Analytics Data Quality ETL Data Architecture Data Pipeline Data Transformation Computer Science Data Engineering Information Technology SQL Python Cloud Computing

Learn the skills this job asks for

See your skill gap for Software Developer →

Free courses, interview questions and practice for each skill.

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
HCLTech logo
HCLTech

Information Technology Services

New Delhi

RecommendedJobs for You

infinite Computer Solution logo

Senior Technical Lead

infinite Computer Solution

India

infinite Computer Solution logo

Senior Technical Lead

infinite Computer Solution

Pune