PySpark Module Lead

Sopra Steria

4 - 6 years

4 - 6 Lacs

noida uttar pradesh india

Posted:2 days ago| Platform: Foundit logo

Apply

Skills Required

data processing deployment management architecture module lead

Work Mode

On-site

Job Type

Full Time

Job Description

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will collaborate closely with our Data Scientists to develop and deploy machine learning models. Proficiency in below listed skills will be crucial in building and maintaining pipelines for training and inference datasets.

Responsibilities:

Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines.
Utilize PySpark for data processing, transformation, and preparation for model training.
Leverage AWS EMR and S3 for scalable and efficient data storage and processing.
Implement and manage ETL workflows using Streamsets for data ingestion and transformation.
Design and construct pipelines to deliver high-quality training and inference datasets.
Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities.
Optimize and fine-tune pipelines for performance, scalability, and reliability.
Ensure IAM policies and permissions are appropriately configured for secure data access and management.
Implement Spark architecture and optimize Spark jobs for scalable data processing.

More Jobs at Sopra Steria

PySpark Module Lead

Bengaluru

4 - 6 yrs

INR 14 - 15 Lacs

SAP PP Senior Functional Analyst

Bengaluru

6 - 8 yrs

INR 8 - 10 Lacs

SAP ABAP UI5 Module Lead

Bengaluru

2 - 6 yrs

INR 15 - 16 Lacs

Full Stack Developer (Java and React)

Noida

8 - 11 yrs

INR 19 - 27 Lacs

Axway B2B Technical Lead

Noida

6 - 8 yrs

INR 8 - 10 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.