Gcp Data Engineer

Role & responsibilities

Key Responsibilities:

• Design, develop, and maintain scalable and efficient 

  data pipelines

   and 

  ETL/ELT workflows

   using GCP services.
• Architect and implement 

  data warehouse solutions

   using 

  BigQuery

  , ensuring optimal performance and cost efficiency.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights.
• Optimize and manage large-scale datasets, ensuring data quality, integrity, and security.
• Develop and enforce best practices for 

  data governance

  , 

  data modeling

  , and 

  data lifecycle management

  .
• Implement 

  streaming data solutions

   using tools like 

  Dataflow

  , 

  Pub/Sub

  , and 

  Apache Beam

  .
• Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
• Lead the migration of on-premise data systems to GCP, ensuring seamless integration and minimal downtime.
• Mentor and guide junior engineers, fostering a culture of continuous learning and innovation.
• Stay updated with the latest advancements in GCP and data engineering technologies, and recommend improvements to existing systems.

Required Skills and Qualifications:

• 10+ years

   of experience in data engineering, with a focus on cloud-based solutions.

• Expertise in GCP services

  , including but not limited to:

• BigQuery

• Dataflow

• Pub/Sub

• Cloud Storage

• Cloud Composer (Airflow)

• Cloud Functions

• Strong proficiency in 

  SQL

   and experience with 

  BigQuery SQL

   for complex queries and performance optimization.
• Hands-on experience with 

  Python

   or 

  Java

   for building data pipelines and automation.
• Deep understanding of 

  data modeling

  , 

  data warehousing

  , and 

  schema design

  .
• Experience with 

  streaming data processing

   and tools like 

  Apache Beam

   or 

  Kafka

  .
• Familiarity with 

  CI/CD pipelines

   and version control systems like 

  Git

  .
• Strong knowledge of 

  data security

   and 

  compliance standards

   (e.g., GDPR, HIPAA).
• Proven experience in 

  migrating on-premise data systems

   to GCP.
• Excellent problem-solving skills and the ability to work in a fast-paced, dynamic environment.
• Strong communication and collaboration skills, with the ability to work effectively with both technical and non-technical stakeholders.