Sr Data Engineer - Apache Dolphin Scheduler

Photon

3 - 8 years

8 - 13 Lacs

pune chennai bengaluru

Posted:2 weeks ago| Platform:

Apply

Skills Required

scala java apache spark python hive continuous integration big data technologies ci/cd pyspark apache pig sql etl pipelines devops hadoop etl big data hbase oozie data processing apache flink data engineering nosql mapreduce kafka sqoop aws

Work Mode

Work from Office

Job Type

Full Time

Job Description

Key Responsibilities:

Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.
Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions.
Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.
Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.
Work closely with data engineers, DevOps, and software teams to enhance workflow automation.
Develop custom plugins and extensions for DolphinScheduler as needed.
Monitor and optimize job execution, resource allocation, and workflow efficiency.
Maintain best practices for CI/CD, version control, and infrastructure automation.

Required Skills & Qualifications:

Strong experience with Apache DolphinScheduler in a production environment.
Proficiency in Java, Python, or Scala for workflow scripting and automation.
Experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.
Understanding of workflow orchestration, DAG execution, and scheduling strategies.
Strong problem-solving skills and ability to work in a fast-paced environment.

More Jobs at Photon

Backend Java Developer

Chennai, Bengaluru, Hyderabad

8 - 12 yrs

INR 8 - 17 Lacs

Java Developer + Angular

Chennai, Bengaluru

4 - 9 yrs

INR 0 - 0 Lacs

Full Stack Developer

Chennai, Bengaluru

6 - 10 yrs

INR 16 - 19 Lacs

SDET Engineer

Chennai

7 - 12 yrs

INR 11 - 12 Lacs

Delivery Manger/Delivery Head

Chennai

14 - 24 yrs

INR 27 - 42 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Java Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.