Senior Pyspark Data Engineer

DATAECONOMY

8 years

0 Lacs

hyderabad telangana india

Posted:2 weeks ago| Platform:

Apply

Skills Required

pyspark data engineering aws apache processing design etl schedule code pipeline resolve architecture workflow extraction git

Work Mode

On-site

Job Type

Full Time

Job Description

Job Title:

Job Summary:

PySpark Data Engineer

Design, develop, and maintain large-scale
ETL pipelines
using
PySpark
and
AWS Glue
.
Orchestrate and schedule data workflows using
Apache Airflow
.
Optimize data processing jobs for performance and cost-efficiency.
Work with large datasets from various sources, ensuring data quality and consistency.
Collaborate with Data Scientists, Analysts, and other Engineers to understand data requirements and deliver solutions.
Write efficient, reusable, and well-documented code following best practices.
Monitor data pipeline health and performance; resolve data-related issues proactively.
Participate in code reviews, architecture discussions, and performance tuning.

Requirements

8+ years of experience in
data engineering
roles.
Strong expertise in
PySpark
for distributed data processing.
Hands-on experience with
AWS Glue
and other AWS data services (S3, Athena, Lambda, etc.).
Experience with
Apache Airflow
for workflow orchestration.
Strong proficiency in
SQL
for data extraction, transformation, and analysis.
Familiarity with data modeling concepts and data lake/data warehouse architectures.
Experience with version control systems (e.g., Git) and CI/CD processes.
Ability to write clean, scalable, and production-grade code.

Benefits
Company Standard Benefits.

More Jobs at DATAECONOMY

AWS PySpark Data Engineer

Hyderabad, Telangana, India

Experience: Not specified

Salary: Not disclosed

Technology Delivery Head - Software Services

Hyderabad, Telangana, India

18 - 22 yrs

Salary: Not disclosed

Senior Specialist Cloud Engineer

Gurgaon, Haryana, India

5.0 - 9.0 yrs

Salary: Not disclosed

System Admin

Hyderabad, Telangana, India

Experience: Not specified

Salary: Not disclosed

Senior Data Scientist

Hyderabad, Telangana, India

9.0 - 9.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.