Data Engineer - GCP

Role Overview

Google Cloud Platform (GCP)

BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Composer

Key Responsibilities

• Data Pipeline Development
• Design, build, and optimize scalable ETL/ELT pipelines using GCP services (Dataflow, Composer, Pub/Sub, BigQuery).
• Ingest structured and unstructured data from multiple sources into cloud storage and warehouse solutions.
• Data Warehousing & Modeling
• Develop efficient and optimized data models in BigQuery.
• Implement partitioning, clustering, and query optimization for performance and cost efficiency.
• Cloud Infrastructure
• Work with GCP-native tools for data integration, storage, and analytics.
• Ensure high availability, security, and compliance of data systems.
• Collaboration & Support
• Partner with data scientists, analysts, and application teams to deliver reliable datasets.
• Troubleshoot issues, monitor pipelines, and ensure SLAs for data delivery.
• Best Practices & Governance
• Implement data quality checks, monitoring, and lineage tracking.
• Follow DevOps and CI/CD practices for data engineering projects.
  

Required Skills & Qualifications

• 3+ years of hands-on experience in data engineering.
• Strong expertise in GCP data services:
• BigQuery (SQL, performance tuning, cost optimization)
• Dataflow (Apache Beam pipelines)
• Cloud Composer (Airflow orchestration)
• Pub/Sub (streaming ingestion)
• Cloud Storage, Dataproc
• Proficiency in SQL and Python for data transformation and automation.
• Good knowledge of ETL/ELT design, data modeling, and warehousing principles.
• Experience with Git, CI/CD pipelines, and version control.
• Understanding of data security, IAM, and compliance in cloud environments.
  

Nice-to-Have Skills

• Familiarity with Terraform/Deployment Manager for IaC (Infrastructure as Code).
• Knowledge of Kafka, Spark, or DBT for advanced data engineering use cases.