GCP Data Engineer

As a GCP Data Engineer/Lead/Architect at TechMango, you will play a crucial role in designing, building, and optimizing data pipelines in the Google Cloud Platform environment. Your expertise in real-time streaming data architectures will be essential in creating scalable and low-latency streaming data pipelines using tools like Pub/Sub, Dataflow (Apache Beam), and BigQuery. Key Responsibilities: - Architect and implement end-to-end streaming data solutions on GCP using Pub/Sub, Dataflow, and BigQuery. - Design real-time ingestion, enrichment, and transformation pipelines for high-volume event data. - Collaborate with stakeholders to understand data requirements and translate them into scalable designs. - Optimize streaming pipeline performance, latency, and throughput. - Build and manage orchestration workflows using Cloud Composer (Airflow). - Drive schema design, partitioning, and clustering strategies in BigQuery for real-time and batch datasets. - Define SLAs, monitoring, logging, and alerting for streaming jobs using Cloud Monitoring, Error Reporting, and Stackdriver. - Ensure robust security, encryption, and access controls across all data layers. - Collaborate with DevOps for CI/CD automation of data workflows using Terraform, Cloud Build, and Git. - Document streaming architecture, data lineage, and deployment runbooks. Required Skills & Experience: - 5+ years of experience in data engineering or architecture. - 3+ years of hands-on GCP data engineering experience. - Strong expertise in Google Pub/Sub, Dataflow (Apache Beam), BigQuery, Cloud Composer (Airflow), and Cloud Storage (GCS). - Solid understanding of streaming design patterns, exactly-once delivery, and event-driven architecture. - Deep knowledge of SQL and NoSQL data modeling. - Hands-on experience with monitoring and performance tuning of streaming jobs. - Experience using Terraform or equivalent for infrastructure as code. - Familiarity with CI/CD pipelines for data workflows. TechMango offers a supportive work environment with benefits including a Badminton Ground, Free Accommodation, Cab Facility for Female Employees, Insurance, GYM, Subsidized Food, Awards and Recognition, and Medical Checkup. Join us as a GCP Data Engineer to drive data excellence and innovation using cutting-edge technology in a dynamic team environment. Apply now for this exciting career opportunity!,