Big Data Engineer

CIEL HR

5 - 10 years

13 - 18 Lacs

pune chennai bengaluru

Posted:23 hours ago| Platform:

Apply

Skills Required

hadoop scala spark aws python airflow pyspark hive big data big data technologies

Work Mode

Hybrid

Job Type

Full Time

Job Description

Position: Scala / Databricks Engineer (Big Data & Spark)

Analyze and understand the current
Apache Spark
platform codebase, including performance nuances.
Optimize and fine-tune Spark jobs for efficiency and high performance.
Work with
large-scale distributed systems
for big data processing and analytics.
Implement best practices for data storage, processing, and retrieval.
Design, build, and maintain ETL pipelines and data workflows.
Handle
Apache Hadoop
development and implementation responsibilities.
Collaborate with data science teams to implement analytics pipelines.
Define and support
data governance
, versioning, and security processes.
Ensure data privacy compliance in coordination with data protection teams.
Work on multiple data sources to uncover insights for business use cases.
Implement streaming solutions using frameworks such as
Kafka
.

Skills & Expertise Required

Must Have

Hands-on experience with
Apache Spark / Spark Streaming
Strong expertise in
Big Data technologies

Good to Have

Python
AWS
(S3, EC2, EMR, IAM, Athena, Lambda)
Kafka
,
Hive

More Jobs at CIEL HR

MSSQL DBA

Chennai, Bengaluru, Mumbai (All Areas)

4 - 9 yrs

INR 8 - 12 Lacs

Contract Construction Company - Bangalore Loc -Project Manager Head

Bengaluru

20 - 30 yrs

INR 15 - 25 Lacs

Credit Underwriter

Bengaluru, Bangalore Rural

4 - 9 yrs

INR 3 - 7 Lacs

Relationship Manager

Jalandhar, Vadodara, Mumbai (All Areas)

1 - 5 yrs

INR 1 - 3 Lacs

Assistant Business Development Manager

Ahmedabad, Delhi / NCR, Mumbai (All Areas)

8 - 13 yrs

INR 6 - 8 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.