Lead GCP Data Engineer

Job Title:

Location:

Experience:

About the Role:

You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data engineers, collaborate with analytics & data science stakeholders, define best-practices, and ensure robust, production-ready pipelines that support business insights and data products.

Key Responsibilities:

• Architect and implement end-to-end data pipelines using BigQuery, PySpark/Spark, Python.
• Use Cloud Composer (Airflow) to orchestrate workflows, manage DAGs, scheduling, dependencies and monitoring.
• Design and optimise BigQuery usage (partitioning, clustering, cost-control, performance tuning).
• Develop and manage large-scale Spark/PySpark jobs for batch and potentially streaming data.
• Ensure cloud-native architecture on GCP: storage (Cloud Storage), compute (Dataproc/Dataflow), orchestration (Composer), security/iam.
• Lead and mentor a team of data engineers: code reviews, best practices, architecture decisions.

Required Skills & Qualifications:

• Minimum ~8 years of experience in data engineering or similar, ideally with lead responsibilities.
• Hands-on experience in GCP services—especially BigQuery (data warehousing, optimisation), Cloud Composer (Airflow) for orchestration.
• Strong experience with PySpark/Spark for large-scale data processing.
• Excellent proficiency in Python (scripting, automation, building frameworks).
• Strong SQL skills (in BigQuery or similar cloud data warehouse), including complex queries, optimisation.
• Experience designing data models, data warehouses/lakes, ingestion pipelines, transformations.
• Experience building/operating CI/CD pipelines, version control (Git), test/monitoring practices.
• Excellent communication and leadership skills—able to mentor, lead architecture discussions and engage stakeholders.

Why Join Us:

• Lead a talented team and influence data engineering architecture & tooling.
• Work on high-impact data platforms that enable business growth.
• Use cutting-edge GCP services and modern data engineering practices.