Data Engineer(Java+Spark)

Persistent

4 - 9 years

4 - 9 Lacs

Bengaluru Karnataka India

Posted:2 weeks ago| Platform: Foundit logo

Apply

Skills Required

Git Bash

Work Mode

On-site

Job Type

Full Time

Job Description

What You'll Do:

Technical Design and implement client requirements.
Build data pipeline.
Perform transformations using Java+spark in data bricks.
Create automated workflows with the help of triggers, Scheduled Jobs in Airflow.
Design and development of Airflow dags to orchestrate the data processing jobs.
Design and development for ensuring timely notifications through email alerts in the event of job failures or critical system issues.
Developing and implementing of code to write the logic data.
Direct customer interaction to understand & gather requirements.
Provision of technical input to customer for analysis and design work.
Support to customer queries.

Expertise You'll Bring:

Proficiency in Java, with a good understanding of its ecosystems. Must have
Experience in Data Engineering Java and Spark, Java is a must. Good knowledge of Spark Architecture
Basic knowledge of Linux / Linux scripting is a must
Transformation and aggregated data from multiple sources. Good Knowledge of Spark Architecture including Spark Core, Spark SQL, RDD, Data Set, and Data Frames. Good to have basic knowledge.
Performance tuning using Optimization techniques, Caching Data in Memory, Broadcast etc. Good to have.
Azure/Cloud DevOps concepts, CI/CD pipeline. Good to have.
Good knowledge on the architecture of a Spark application. Good to have.
Knows the concepts of deployment pipelines. Good to have.
Hands-on experience with git bash. Must have
Has worked with scrum methodology. Familiar with ceremonies and ways of working. Good to have.
Able to communicate with the end clients via email with good email etiquette and via phone.
Understanding business needs, identifying the right data sources, developing scalable and reliable data pipelines to ensure a smooth process

More Jobs at Persistent

Dot Net Developer

Chennai, Pune, Bengaluru

6 - 10 yrs

INR 14 - 22 Lacs

Power BI Developer

Pune

8 - 11 yrs

INR 27 - 37 Lacs

Junior Power BI Developer

Pune

3 - 7 yrs

INR 7 - 14 Lacs

Java Developer

Bengaluru

5 - 10 yrs

INR 15 - 30 Lacs

Java Programmer Developer

Bengaluru

5 - 10 yrs

INR 14 - 22 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.