GCP Data Engineer with Big Query

6 - 10 years

27 - 42 Lacs

Posted:18 hours ago| Platform: Naukri logo

Apply

Work Mode

Work from Office

Job Type

Full Time

Job Description

Skill - GCP , BigQuery, Python

Location - Coimbatore

Job Summary

Data Engineer with a focus on Google Cloud Platform (GCP).

Hands-on experience with GCP services: Dataflow Pub/Sub Cloud Storage Cloud Composer.

Expertise in workflow orchestration using Apache Airflow.

Experience with cloud SQL or other relational databases (PostgreSQL preferred).

Solid understanding of data modeling ETL processes and streaming/batch data pipelines.

Familiarity with DevOps practices CI/CD and containerization (Docker Kubernetes).


 

Responsibilities

Key Responsibilities and Skills

 

  • Build Data Pipelines on GCP: They need hands-on experience with core GCP services used for data processing.
  • Dataflow: For building both batch and streaming data pipelines.
  • Pub/Sub: For real-time messaging and event ingestion.
  • Cloud Storage: For storing raw and processed data.
  • Cloud Composer: A managed service for Apache Airflow used for orchestrating complex data workflows.
  • Orchestrate Workflows: A key requirement is expertise in Apache Airflow. This tool is essential for scheduling monitoring and managing data workflows often represented as Directed Acyclic Graphs (DAGs).
  • Manage Databases: The role requires experience with Cloud SQL or other relational databases with a preference for PostgreSQL. This suggests they will be working with structured data storage.
  • Understand Data Concepts: A strong foundation in the theoretical side of data engineering is crucial.
  • Data Modeling: Designing the structure of databases to meet business needs.
  • ETL Processes: The classic Extract Transform Load pipeline for moving and processing data.
  • Streaming/Batch Data Pipelines: Understanding the differences and how to build both types of pipelines.
  • Apply DevOps Practices: Familiarity with modern software development practices is a plus.
  • CI/CD: Continuous Integration and Continuous Delivery for automating code deployment.
  • Containerization: Using technologies like Docker and Kubernetes to package and deploy applications consistently across different environments.

Mock Interview

Practice Video Interview with JobPe AI

Start DevOps Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Cognizant logo
Cognizant

IT Services and IT Consulting

Teaneck New Jersey

RecommendedJobs for You

mumbai, delhi / ncr, bengaluru

gandhinagar, visakhapatnam, warangal, hyderabad, chennai, coimbatore, aurangabad, noida, bhubaneswar, kolkata, mumbai, indore, nagpur, chandigarh, nashik, new delhi, mysuru, pune, ahmedabad, gurugram, bengaluru, vadodara

mumbai, gurugram, bengaluru