Data Engineer - GCP (Google Cloud Platform)

Experience Level: 46 years

Note -

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines using GCP services such as: BigQuery, Cloud Dataflow, Pub/Sub, Cloud Storage, Cloud Composer (Airflow).


• Implement data ingestion from diverse sources (e.g., APIs, databases, files) into cloud data lake/warehouse.


• Optimize data processing workflows for performance, reliability, and cost efficiency.


• Develop and manage ETL/ELT processes using tools like Dataform, dbt, or custom scripts.


• Collaborate with data scientists, analysts, and stakeholders to deliver clean, well-documented datasets.


• Implement data quality, governance, and observability practices (e.g., Great Expectations, Data Catalog).


• Monitor, debug, and improve the performance of data pipelines and batch/streaming jobs.


• Ensure security, privacy, and compliance standards are met in all data engineering activities.

Required Qualifications

• 4-6 years of experience in data engineering or a related field.


• Hands-on experience with Google Cloud Platform (GCP), particularly:


• BigQuery, Cloud Dataflow (Apache Beam), Pub/Sub, Cloud Storage, Composer (Airflow).


• Strong programming skills in Python, SQL, and optionally Java or Scala.


• Proficient in building and maintaining ETL/ELT pipelines in cloud environments.


• Understanding of data modeling, data warehousing, and batch vs streaming architectures.


• Familiarity with CI/CD practices, version control (Git), and infrastructure-as-code (e.g., Terraform, Deployment Manager).

Preferred Qualifications

• Experience with tools like dbt, Dataform, Looker, or similar.


• Knowledge of data security best practices, including IAM, encryption, and audit logging.


• Exposure to real-time data processing and event-driven architectures.


• GCP certifications such as:


• Professional Data Engineer


• Associate Cloud Engineer