Gcp Data Engineer

Datametica

4 - 9 years

10 - 20 Lacs

pune bengaluru

Posted:14 hours ago| Platform:

Apply

Skills Required

pyspark gcp bigquery pubsub dataproc

Work Mode

Hybrid

Job Type

Full Time

Job Description

Key Responsibilities:

Design, build, and optimize scalable ETL/ELT pipelines using
PySpark
on both Hadoop and GCP environments.
Utilize Hadoop ecosystem components including
Hive
,
HDFS
,
YARN
, and
Oozie
for data storage, processing, and workflow orchestration.
Implement and manage GCP data services such as
BigQuery
,
Cloud Storage
,
Dataflow
, and
Cloud Composer
to support data ingestion, transformation, and analytics workloads.
Ensure the performance, scalability, and reliability of data pipelines to meet business and operational requirements.
Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data needs and deliver high-quality data solutions.
Monitor and troubleshoot data pipelines, proactively addressing issues to minimize downtime.

Required Qualifications:

Bachelors degree in Computer Science, Information Technology, or a related field (or equivalent work experience).
4 to 10 years of hands-on experience in data engineering or related roles.
Proficiency in
PySpark
programming and writing complex
SQL
queries.
Strong experience working with Hadoop ecosystem tools:
Hive
,
HDFS
,
YARN
, and
Oozie
.
Practical knowledge of Google Cloud Platform data services including
BigQuery
,
Dataflow
,
Cloud Storage
, and
Cloud Composer
.
Demonstrated ability to build scalable and reliable data pipelines in distributed environments.
Excellent analytical, problem-solving, and communication skills.

More Jobs at Datametica

Talent Acquisition Executive

Pune

1.0 - 3.0 yrs

INR 1 - 3 Lacs

Sr. SQL DBA

Hyderabad, Pune, Bengaluru

6.0 - 11.0 yrs

INR 15 - 25 Lacs

AI/ML Lead

Pune

7.0 - 10.0 yrs

INR 15 - 30 Lacs

Senior Data Engineer

Pune, Bengaluru

5.0 - 10.0 yrs

INR 8 - 18 Lacs

Solution Architect (AI/ML Specialization)

Pune

10.0 - 18.0 yrs

INR 45 - 65 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.