Senior Big Data Engineer

5 years

0 Lacs

Posted:4 days ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Job Role:

Location:

Experience:


Role Overview

Big Data Engineer


Key Responsibilities

  • Design, develop, and optimize

    data pipelines

    processing

    terabytes of structured and unstructured data per day

    .
  • Build

    real-time and batch data processing systems

    using Spark, Kafka, and Hadoop ecosystem tools.
  • Manage

    data ingestion, transformation, and storage

    for analytics, ML, and reporting needs.
  • Work with

    Data Scientists, ML Engineers, and Analysts

    to deliver scalable, production-ready datasets.
  • Implement

    data quality, lineage, and observability frameworks

    to ensure reliability and consistency.
  • Optimize

    ETL performance, cluster utilization, and query execution

    across distributed environments.
  • Manage

    data workflows on the cloud

    (AWS, GCP, or Azure) for speed, cost, and scalability.


Must-Have Skills

  • Deep expertise with

    Apache Spark, Kafka, Hadoop (HDFS, Hive, YARN), Airflow

    .
  • Strong programming skills in

    Python, Scala, or Java

    .
  • Proven experience handling

    large-scale, high-velocity data (TBs/day)

    in production systems.
  • Strong understanding of

    distributed computing, data partitioning, and cluster performance tuning

    .
  • Hands-on with

    cloud-native data tools

    (AWS EMR, Glue, Redshift, GCP BigQuery, or Azure Databricks).
  • Familiarity with

    data lake/lakehouse architectures

    and

    schema evolution

    .


Good to have

  • Experience with

    real-time streaming

    (Flink, Kinesis, Pulsar) and

    data lake formats

    (Delta, Iceberg, Hudi).
  • Understanding of

    data governance, cataloging, and lineage

    .
  • Exposure to

    ML pipelines or feature store engineering

    .


📩 𝐀𝐩𝐩𝐥𝐲 𝐧𝐨𝐰: Send your CV to 𝐩𝐫𝐢𝐭𝐢.𝐫𝐚𝐭𝐡𝐨𝐫𝐞@𝐯𝐚𝐫𝐚𝐢𝐬𝐲𝐬.𝐜𝐨𝐦

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You

pune, maharashtra, india