Big Data Engineer (8-16 Years)

9 - 14 years

Noida Hyderabad Bengaluru

Posted:10 hours ago

Job Description

Position:

About the Role

We are looking for experienced Big Data Developers to design, build, and optimize large-scale data processing systems and analytics platforms. The ideal candidate will possess strong expertise in Big Data technologies, distributed computing, cloud platforms, and data engineering practices to support enterprise data-driven initiatives.

Key Responsibilities

  • Design, develop, and maintain scalable Big Data solutions for processing and analyzing large volumes of structured and unstructured data.

  • Build and optimize data pipelines, ETL/ELT processes, and real-time streaming applications.

  • Develop data ingestion frameworks from multiple sources including APIs, databases, files, and streaming platforms.

  • Implement batch and real-time data processing solutions using Big Data technologies.

  • Collaborate with Data Architects, Data Scientists, Business Analysts, and stakeholders to understand business requirements.

  • Ensure data quality, integrity, governance, and security across the data ecosystem.

  • Optimize data processing performance and troubleshoot production issues.

  • Participate in architecture reviews, code reviews, and technical discussions.

  • Support cloud migration and modernization initiatives.

  • Develop and maintain technical documentation and operational procedures.

Required Skills

Big Data Technologies

  • Strong hands-on experience with:

    • Hadoop Ecosystem

    • Apache Spark (Core, SQL, Streaming)

    • Hive

    • HDFS

    • YARN

  • Experience developing distributed data processing applications.

Data Engineering

  • Expertise in ETL/ELT development.

  • Strong knowledge of data modeling and data warehousing concepts.

  • Experience with large-scale data transformation and migration projects.

  • Understanding of data governance and data quality frameworks.

Programming Skills

  • Strong proficiency in:

    • Python

    • Scala

    • Java

    • SQL

  • Experience with scripting and automation.

Streaming Technologies

  • Experience with:

    • Apache Kafka

    • Spark Streaming

    • Apache Flink (Preferred)

    • Apache NiFi

Cloud Technologies

  • Experience with one or more cloud platforms:

    • AWS

    • Azure

    • Google Cloud Platform (GCP)

Database Technologies

  • Strong experience with:

    • SQL Server

    • Oracle

    • PostgreSQL

    • MySQL

    • MongoDB

    • Cassandra (Preferred)

Tools & Technologies

  • Apache Spark

  • Hadoop

  • Hive

  • Kafka

  • Scala

  • Databricks

  • Snowflake

  • AWS

  • Azure

Notice Period:

Keywords:

Provide your feedback on BizChat

Skills Required

Hive SCALA Hadoop Big Data Apache Spark Pyspark Sqoop Design Patterns

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Hybrid

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Big Data Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Infosys logo
Infosys

IT Services and IT Consulting

Bangalore Karnataka

RecommendedJobs for You