Big Data Engineer

Nice Software Solutions Pvt. Ltd.

8 - 12 years

0 Lacs

pune maharashtra india

Posted:9 hours ago| Platform:

Apply

Skills Required

data pyspark design hive hadoop tuning agile schedule apache resolve reliability sql collaborative workflow orchestration airflow devops spark

Work Mode

On-site

Job Type

Full Time

Job Description

Big Data Engineer (PySpark)

Location:

Experience:

Employment Type:

Job Overview

Big Data Engineer

Key Responsibilities

Design, develop, and maintain data pipelines using
PySpark
and related Big Data technologies.
Work with
HDFS, Hive, Sqoop
, and other tools in the Hadoop ecosystem.
Write efficient
HiveQL and SQL
queries to handle large-scale datasets.
Perform
performance tuning and optimization
of distributed data systems.
Collaborate with cross-functional teams in an
Agile environment
to deliver high-quality solutions.
Manage and schedule workflows using
Apache Airflow
or
Oozie
.
Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.

Required Skills

Proven experience in
Big Data Engineering
with a focus on PySpark.
Strong knowledge of
HDFS, Hive, Sqoop
, and related tools.
Proficiency in
SQL/HiveQL
for large datasets.
Expertise in
performance tuning and optimization
of distributed systems.
Familiarity with
Agile methodology
and collaborative team practices.
Experience with workflow orchestration tools (
Airflow/Oozie
).
Strong
problem-solving, analytical, and communication
skills.

Good to Have

Knowledge of
data modeling
and
data warehousing
concepts.
Exposure to
DevOps practices
and
CI/CD pipelines
for data engineering.
Experience with other Big Data frameworks such as
Spark Streaming
or
Kafka
.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.