Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in hyderabad
>
Zorba AI
>
Lead Data Engineer

Lead Data Engineer

Zorba AI

16 years

0 Lacs

hyderabad telangana india

Posted:20 hours ago| Platform:

Apply

Skills Required

data engineering python pyspark aws design etl analytics development integration checks monitoring sql scalability storage redshift code tuning mentoring governance reliability efficiency programming scripting database querying joins query optimization spark git github bitbucket workflow orchestration apache airflow docker kubernetes containerization devops strategies architecture datadog

Work Mode

On-site

Job Type

Full Time

Job Description

Primary Job Title:

Data Engineering Lead

About The Opportunity

We are seeking a highly skilled

Lead Data Engineer

with strong expertise in

Python, Pandas, PySpark, AWS, and SQL

to design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments.Role & Responsibilities

Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform.
Implement data quality frameworks, validation checks, and monitoring solutions using Python and SQL.
Optimize PySpark jobs for performance and scalability in a distributed computing environment (AWS EMR/Glue).
Develop reusable ETL frameworks using PySpark and Pandas for data transformation and analytics.
Manage and maintain cloud-based infrastructure and data storage (AWS S3, Redshift, Lambda, Glue, Athena).
Collaborate with data scientists, analysts, and stakeholders to provide clean, structured, and accessible datasets.
Oversee code reviews, performance tuning, and mentoring junior data engineers.
Establish best practices for data governance, version control, and CI/CD integration.
Troubleshoot production data issues and ensure system reliability, availability, and efficiency.

Must-Have Skills

Programming: Python (advanced scripting, Pandas, PySpark)
Cloud: AWS (S3, Glue, Lambda, Redshift, Athena, EMR)
Database / Querying: Advanced SQL (Joins, Window Functions, Query Optimization)
Big Data Tools: PySpark, Spark SQL
ETL Development: Data ingestion, transformation, and validation pipelines
Version Control: Git / GitHub / Bitbucket
Workflow Orchestration: Apache Airflow or equivalent (preferred)

Good-to-Have Skills

Experience with Docker or Kubernetes for containerization.
Familiarity with CI/CD pipelines and DevOps practices.
Exposure to data modeling, schema design, and partitioning strategies.
Understanding of data lake and data warehouse architecture.
Knowledge of monitoring tools (CloudWatch, Datadog, or Prometheus).

Qualifications

Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
11–16 years of total experience with at least 3–4 years leading data engineering projects.
Proven track record in handling large-scale data pipelines and cloud migrations.

Skills: python,aws,sql,pyspark

More Jobs at Zorba AI

SSAS +MDX-Data Engineer_5+yrs

Noida, Uttar Pradesh, India

Experience: Not specified

Salary: Not disclosed

SSAS +MDX-Data Engineer_5+yrs

New Delhi, Delhi, India

Experience: Not specified

Salary: Not disclosed

SSAS +MDX-Data Engineer_5+yrs

Delhi, India

Experience: Not specified

Salary: Not disclosed

SSAS +MDX-Data Engineer_5+yrs

Gurugram, Haryana, India

Experience: Not specified

Salary: Not disclosed

SSAS +MDX-Data Engineer_5+yrs

Faridabad, Haryana, India

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.