Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in pune
>
ITC Infotech
>
Lead Data Engineer

Lead Data Engineer

ITC Infotech

7 years

0 Lacs

pune maharashtra india

Posted:20 hours ago| Platform:

Apply

Skills Required

data timing drive rpc learning training ai design extract ml workflow support model spark storage databricks auto scaling monitoring pipeline apache vsam python pyspark programming github

Work Mode

On-site

Job Type

Full Time

Job Description

Lead Data Engineer

Location: Banglore/Pune (Hybrid)

Mode: Hybrid

Shift Timing: 2 PM to 11 PM

Experience: 7-12 years.

Purpose:

Technical Lead

Key Responsibilities

Mainframe Data Ingestion:
Design and implement scalable pipelines to extract, parse, and ingest RPC logs and technical attributes from mainframe systems into Delta Lake on Databricks.
ML-Ready Data Engineering:
Transform and structure data for time-series modelling and anomaly detection across thousands of models.
ML Workflow Integration:
Collaborate with ML engineers to ensure data pipelines support SARIMA, ANN, and other model types; enable automated retraining and scoring.
Performance Optimization:
Tune Spark jobs, Delta Lake storage, and cluster configurations for billions of records and real-time aggregation.
FinOps & Cost Control:
Monitor and optimize Databricks resource usage; implement auto-scaling and cost-aware job scheduling.
Monitoring & Alerting:
Integrate Databricks-native alerting for pipeline health, data anomalies, and job failures.

Required Skills

Strong hands-on experience with
Databricks
,
Apache Spark
,
Delta Lake
, and
MLflow
Proven expertise in
mainframe data integration
(e.g., SMF, RPC logs, VSAM, DB2)
Strong Python and PySpark programming skills

Preferred Skills

Databricks experience with
CI/CD tools
GitHub Actions
Knowledge of
FinOps principles
for cloud cost optimization

More Jobs at ITC Infotech

Salesforce Marketing Cloud Developer

Bengaluru

4 - 9 yrs

INR 6 - 13 Lacs

Open Text - VIM - Product Lead / Manager

Kolkata, West Bengal, India

10 - 10 yrs

Salary: Not disclosed

Salesforce Marketing Cloud Developer

Bengaluru

4 - 8 yrs

INR 8 - 13 Lacs

Salesforce Developer

Bengaluru

3 - 8 yrs

INR 7 - 17 Lacs

Trade Finance Functional Tester

Bengaluru

5 - 10 yrs

INR 10 - 17 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.