Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Jobs

Interviews

EMR_Spark SME

Lorven Technologies Private Limited

5 - 10 years

3 - 7 Lacs

Bengaluru

Posted:2 weeks ago| Platform:

Apply

Skills Required

resource management continuous integration spark streaming spark ci cd pipeline python scala glue version control amazon redshift big data technologies ci/cd spark core java aws emr athena aws big data database system architecture

Work Mode

Work from Office

Job Type

Full Time

Job Description

Job Title:EMR_Spark SMEExperience:5-10 YearsLocation:Bangalore
:
Technical
Skills:

5+ years of experience in big data technologies with hands-on expertise in AWS EMR and Apache Spark.

Proficiency in Spark Core, Spark SQL, and Spark Streaming for large-scale data processing.

Strong experience with data formats (Parquet, Avro, JSON) and data storage solutions (Amazon S3, HDFS).

Solid understanding of distributed systems architecture and cluster resource management (YARN).

Familiarity with AWS services (S3, IAM, Lambda, Glue, Redshift, Athena).

Experience in scripting and programming languages such as Python, Scala, and Java.

Knowledge of containerization and orchestration (Docker, Kubernetes) is a plus.

Architect and develop scalable data processing solutions using AWS EMR and Apache Spark.

Optimize and tune Spark jobs for performance and cost efficiency on EMR clusters.

Monitor, troubleshoot, and resolve issues related to EMR and Spark workloads.

Implement best practices for cluster management, data partitioning, and job execution.

Collaborate with data engineering and analytics teams to integrate Spark solutions with broader data ecosystems (S3, RDS, Redshift, Glue, etc.).

Automate deployments and cluster management using infrastructure-as-code tools like CloudFormation, Terraform, and CI/CD pipelines.

Ensure data security and governance in EMR and Spark environments in compliance with company policies.

Provide technical leadership and mentorship to junior engineers and data analysts.

Stay current with new AWS EMR features and Spark versions to recommend improvements and upgrades.

Requirements and Skills

Performance tuning and optimization of Spark jobs.

Problem-solving skills with the ability to diagnose and resolve complex technical issues.

Strong experience with version control systems (Git) and CI/CD pipelines.

Excellent communication skills to explain technical concepts to both technical and non-technical audiences.

Qualification:

Education qualificationB.Tech, BE, BCA, MCA, M. Tech or equivalent technical degree from a reputed college.

Certifications:

AWS Certified Solutions Architect Associate/Professional

AWS Certified Data Analytics Specialty

More Jobs at Lorven Technologies Private Limited

EWT / Portal - Full Stack - Java,Java, Spring Boot, Maven, Angular

Bengaluru

5.0 - 10.0 yrs

INR 5 - 9 Lacs

Artificial Intelligence Engineer

Bengaluru

8.0 - 12.0 yrs

INR 8 - 12 Lacs

SFMC Developer

Chennai, Tiruchirapalli

5.0 - 10.0 yrs

INR 1 - 4 Lacs

Delivery Module Lead

Pune

6.0 - 8.0 yrs

INR 7 - 12 Lacs

Sr AWS Data Engineer

Hyderabad, Pune, Gurugram

4.0 - 6.0 yrs

INR 2 - 6 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

Lorven Technologies Private Limited

Information Technology

N/A

Login to

Please Verify Your Phone or Email

Confirm Action

Search

Profile

Upskill and Grow with AI

EMR_Spark SME