PySpark Developer

Apex 2000

4 - 7 years

4 - 7 Lacs

Chennai Tamil Nadu India

Posted:1 day ago| Platform: Foundit logo

Apply

Skills Required

PySpark framework Athena

Work Mode

On-site

Job Type

Full Time

Job Description

PySpark Developer

Roles & Responsibilities:

Utilize strong working experience in
Python programming
to develop data processing solutions.
Apply expertise with the
PySpark framework
for big data processing.
Work extensively with
AWS Glue and PySpark
for building and managing data pipelines.
Use popular Python libraries such as
pandas, numpy, and joblib
to enhance data manipulation and processing.
Conduct
parallel batch processing with Python
to handle large-scale data.
Work with
AWS Batch and Step Functions
for orchestrating and managing data workflows.
Utilize knowledge of
AWS ECS, ECR, and Docker
for containerization and deployment.
Leverage knowledge of
CloudWatch, Athena
, and other AWS services for monitoring, analytics, and data management.
Collaborate with data scientists and other engineers to understand requirements and deliver efficient solutions.
Troubleshoot and optimize data processing jobs for performance and cost efficiency.

Skills Required:

Strong working experience in
Python programming
.
Expertise with
PySpark framework
.
Must have experience on
AWS Glue and PySpark
.
Strong experience with using
pandas, numpy, joblib
, and other popular libraries.
Good working experience with
parallel batch processing with Python
.
Good working experience on
AWS Batch and Step Functions
.
Good experience on
AWS ECS, ECR, Docker
.
Good knowledge on
CloudWatch, Athena
, and other AWS services.
Strong problem-solving and analytical skills.
Excellent communication and collaboration abilities.

QUALIFICATION:

Bachelor's degree in Computer Science, Information Technology, or a related field, or equivalent practical experience.

More Jobs at Apex 2000

SAP EWM Professional

Bengaluru

5.0 - 9.0 yrs

INR 7 - 11 Lacs

SQL Server Expert

Delhi, Delhi, India

5.0 - 8.0 yrs

INR 5 - 8 Lacs

SAP EWM Professional

Bengaluru, Karnataka, India

10.0 - 15.0 yrs

INR 10 - 15 Lacs

Solutions Architect

Pune, Maharashtra, India

5.0 - 10.0 yrs

INR 5 - 10 Lacs

Murex

Bengaluru, Karnataka, India

3.0 - 6.0 yrs

INR 3 - 6 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.