Data Engineer (Python + PySpark + AWS)

Staffice Global Private Limited

5 - 9 years

5 - 14 Lacs

Gurgaon Haryana India

Posted:2 days ago| Platform: Foundit logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Data Engineer

Key Responsibilities

Design and implement end-to-end data engineering solutions using a combination of
Python
and
AWS data services
like
Glue, Lambda, and EMR
.
Develop and maintain scalable
ETL (Extract, Transform, Load)
pipelines.
Utilize
PySpark
for efficient big data processing and transformation.
Work with
Kafka
for real-time data ingestion and stream processing.
Optimize and fine-tune data pipelines for performance, scalability, and cost-efficiency on the
AWS
platform.
Implement data security measures and contribute to data governance practices.
Collaborate with cross-functional teams to understand data requirements and translate them into scalable data architectures.
Troubleshoot and resolve complex data workflow issues.

Skills

Required Skills:

Strong expertise in
Python
and
SQL
for data manipulation and pipeline development.
Hands-on experience with
PySpark
for big data processing.
Deep understanding of
AWS data services
such as
Redshift, Glue, S3, Lambda, Kinesis, Athena, and DynamoDB
.
Experience with
Kafka
for real-time data ingestion.
Strong experience in
data modeling, schema design, and ETL best practices
.
Excellent problem-solving skills with the ability to debug complex data workflows.

Preferred Skills:

Familiarity with
infrastructure as code (IaC)
tools like
Terraform or CloudFormation
.

Qualifications

A minimum of
5 years of experience
in data engineering, data warehousing, and big data processing.

More Jobs at Staffice Global Private Limited

Automation test Lead with Chatbot

Coimbatore, Tamil Nadu, India

7.0 - 11.0 yrs

INR 5 - 8 Lacs

ReactJS Developer

Bengaluru, Karnataka, India

4.0 - 8.0 yrs

INR 5 - 10 Lacs

Software Engineer - Oracle

Chennai, Tamil Nadu, India

4.0 - 10.0 yrs

INR 4 - 10 Lacs

Business Development Manager

Chennai, Tamil Nadu, India

5.0 - 10.0 yrs

INR 5 - 10 Lacs

Senior Functional Analyst - HCM

Chennai, Tamil Nadu, India

6.0 - 8.0 yrs

INR 6 - 8 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.