ETL_GCP_Bigquery

Job Description

Develop, implement, and optimize ETL/ELT pipelines for processing large datasets efficiently. • Work extensively with BigQuery for data processing, querying, and optimization. • Utilize Cloud Storage, Cloud Logging, Dataproc, and Pub/Sub for data ingestion, storage, and event-driven processing. • Perform performance tuning and testing of the ELT platform to ensure high efficiency and scalability. • Debug technical issues, perform root cause analysis, and provide solutions for production incidents. • Ensure data quality, accuracy, and integrity across data pipelines. • Collaborate with cross-functional teams to define technical requirements and deliver solutions. • Work independently on assigned tasks while maintaining high levels of productivity and efficiency. Skills Required: • Proficiency in SQL and PL/SQL for querying and manipulating data. • Experience in Python for data processing and automation. • Hands-on experience with Google Cloud Platform (GCP), particularly: o BigQuery (must-have) o Cloud Storage o Cloud Logging o Dataproc o Pub/Sub • Experience with GitHub and CI/CD pipelines for automation and deployment. • Performance tuning and performance testing of ELT processes. • Strong analytical and debugging skills to resolve data and pipeline issues efficiently. • Self-motivated and able to work independently as an individual contributor. • Good understanding of data modeling, database design, and data warehousing concepts. Role & responsibilities Preferred candidate profile