GCP AI Data Engineer | 3 +Years

GCP AI Data Engineer

Key Responsibilities:

• Design and implement

  scalable data architectures

  on GCP, including data lakes, data warehouses, and real-time processing systems.
• Develop and optimize

  ETL/ELT pipelines

  using Python, SQL, Airflow, and streaming technologies.
• Utilize

  Vertex AI and ML services

  to operationalize machine learning models and support AI/ML workflows (e.g., data labeling, classification, document parsing).
• Architect and manage

  historical and incremental data loads

  , while continuously refining architecture for efficiency.
• Manage and optimize

  data storage, partitioning, and clustering strategies

  using BigQuery, Pub/Sub, Dataproc, and Cloud Storage.
• Automate infrastructure and deployments with

  Infrastructure as Code (IaC)

  tools like Terraform and implement CI/CD practices (e.g., Tekton).
• Collaborate with data scientists, data engineers, and business stakeholders to ensure solutions align with business needs, security, and governance standards.
• Monitor, troubleshoot, and resolve issues in data pipelines related to

  performance, reliability, and cost-effectiveness

  .
• Create and maintain documentation for data processes, pipeline designs, and system architecture.

Skills Required:

• Strong expertise in

  Google Cloud Platform (GCP)

  services including

  BigQuery, Dataflow, Data Fusion, Dataproc, Pub/Sub, and Cloud Storage

• Hands-on experience in

  workflow orchestration and automation tools

  such as

  Apache Airflow

• Proficiency in

  data modeling, ETL/ELT pipeline development

  , and

  SQL/Python

  programming
• Solid understanding of

  relational databases

  with experience in

  Postgres

• Practical knowledge of

  Infrastructure as Code (IaC)

  using

  Terraform

  for deployment automation
• Ability to design and optimize

  scalable data architectures

  for enterprise data lakes and data warehouses