Lead GCP Data Engineer

As a Lead Data Engineer with 8+ years of experience in data engineering/analytics engineering and previous lead responsibilities, your role will involve leading the design, build, and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data engineers, collaborate with analytics & data science stakeholders, define best practices, and ensure robust, production-ready pipelines that support business insights and data products. Key Responsibilities: - Architect and implement end-to-end data pipelines using BigQuery, PySpark/Spark, Python. - Use Cloud Composer (Airflow) to orchestrate workflows, manage DAGs, scheduling, dependencies, and monitoring. - Design and optimize BigQuery usage (partitioning, clustering, cost-control, performance tuning). - Develop and manage large-scale Spark/PySpark jobs for batch and potentially streaming data. - Ensure cloud-native architecture on GCP: storage (Cloud Storage), compute (Dataproc/Dataflow), orchestration (Composer), security/iam. - Lead and mentor a team of data engineers: code reviews, best practices, architecture decisions. Required Skills & Qualifications: - Minimum ~8 years of experience in data engineering or similar, ideally with lead responsibilities. - Hands-on experience in GCP services, especially BigQuery (data warehousing, optimization), Cloud Composer (Airflow) for orchestration. - Strong experience with PySpark/Spark for large-scale data processing. - Excellent proficiency in Python (scripting, automation, building frameworks). - Strong SQL skills (in BigQuery or similar cloud data warehouse), including complex queries, optimization. - Experience designing data models, data warehouses/lakes, ingestion pipelines, transformations. - Experience building/operating CI/CD pipelines, version control (Git), test/monitoring practices. - Excellent communication and leadership skills able to mentor, lead architecture discussions, and engage stakeholders. In this role, you will have the opportunity to lead a talented team and influence data engineering architecture & tooling. You will work on high-impact data platforms that enable business growth while using cutting-edge GCP services and modern data engineering practices.,