Apache beam software engineer

4 - 9 years

10 - 20 Lacs

Posted:1 week ago| Platform: Naukri logo

Apply

Work Mode

Remote

Job Type

Full Time

Job Description

Job Title:

Job Summary:

Software Engineer

Key Responsibilities:

  • Design and develop

    framework-level components

     using

    Apache Beam

    ,

    GCP Dataflow

    , and

    Dataproc

    .
  • Build scalable, reusable libraries and abstractions in

    Python

     or

    Java

     for distributed data processing.
  • Work closely with architects to implement best practices for designing high-performance data frameworks.
  • Ensure software reliability, maintainability, and testability through strong coding and automation practices.
  • Participate in code reviews, architectural discussions, and performance tuning initiatives.
  • Contribute to internal tooling or SDK development for data engineering platforms.

Required Skills:

  • 4 to 6 years of experience as a

    Software Engineer

     working on distributed systems or data processing frameworks.
  • Strong programming skills in

    Java

     and/or

    Python

    .
  • Deep experience with

    Apache Beam

     and

    GCP Dataflow

    .
  • Hands-on experience with

    GCP Dataproc

    , especially for building scalable custom batch or streaming jobs.
  • Solid understanding of

    streaming vs batch

     processing concepts.
  • Familiarity with

    CI/CD pipelines

    ,

    GitHub

    , and test automation.

Preferred Skills:

  • Experience with

    workflow orchestration tools

     such as

    Airflow (Composer)

    .
  • Exposure to

    Pub/Sub

     and

    BigQuery

     (from a system integration perspective).
  • Understanding of

    observability

    ,

    logging

    , and

    error-handling

     in distributed data pipelines.
  • Experience building internal libraries, SDKs, or tools to support data teams.

Tech Stack:

  • Cloud:

     GCP (Dataflow, Dataproc, Pub/Sub, Composer)
  • Programming:

     Java, Python
  • Frameworks:

     Apache Beam
  • DevOps:

     GitHub, CI/CD (Cloud Build, Jenkins)
  • Focus Areas:

     Framework/library development, scalable distributed data processing, component-based architecture

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You

Pune, Chennai, Bengaluru