Big Data Engineer

4 - 9 years

16 - 31 Lacs

Chennai Bengaluru

Posted:19 hours ago

Job Description

**Job Title:** Data Engineering Hadoop Developer (3-8 Years Experience)

Core Python, Pyspark, SQL. Mandatory background/Hands on exp on Apache Iceberg

**Job Summary**

Seeking a Hadoop Developer with 3-8 years of experience in big data engineering to design, build, and optimize scalable data pipelines on Hadoop ecosystem technologies. The role involves batch/stream processing, data integration, performance tuning, and close collaboration with business teams.

**Key Responsibilities**

1. Design and develop robust ETL/data pipelines using Hadoop ecosystem tools.

2. Build and optimize large-scale data processing jobs using Spark and/or Hive.

3. Develop workflows using Airflow.

4. Write complex SQL/HiveQL for data transformation and reporting needs.

5. Ingest structured and unstructured data from multiple sources (Kafka, Sqoop, APIs, files).

6. Ensure data quality, lineage, and governance standards are followed.

7. Monitor, troubleshoot, and tune jobs for performance and scalability.

8. Create technical documentation and follow coding best practices.

**Required Skills**

1. 3-8 years of hands-on experience in Hadoop/big data development.

2. Strong knowledge of HDFS, Hive, Spark,

3. String knowledge in Python/Scala

4. Strong SQL and data modelling fundamentals.

5. Experience with workflow orchestration tools (Airflow).

6. Good understanding of Linux, shell scripting, and distributed systems.

8. Familiarity with version control (Git) and Agile delivery practices.

Skills Required

Big Data Hive Hadoop Pyspark Hdfs Python Spark Data

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Hybrid

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Big Data Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Tech Mahindra logo
Tech Mahindra

Information Technology & Services

Noida

RecommendedJobs for You

SP Software logo

Bengaluru

Bristlecone logo

Bengaluru

Athena Consultancy Services logo

Big Data Engineer

Athena Consultancy Services

Pune

Evoke HR logo

Chennai, Bengaluru

B

Big Data Engineer

BRISKWIN IT SOLUTIONS

Chennai

HNM Systems logo

Hyderabad

Ambiquest Hyderabad logo

Big Data Engineer

Ambiquest Hyderabad

Chennai, Bengaluru

dunnhumby logo

Gurugram