We are looking for an experienced GCP Data Engineer with a minimum of 5+ years of professional experience in data engineering, cloud-based data solutions, and large-scale distributed systems. This role is fully remote and requires a hands-on professional who can design, build, and optimize data pipelines and solutions on Google Cloud Platform (GCP).
Key Responsibilities
- Architect, design, and implement highly scalable data pipelines and ETL workflows leveraging GCP services.
- Develop and optimize data ingestion, transformation, and storage frameworks to support analytical and operational workloads.
- Work extensively with BigQuery, Dataflow, Pub/Sub, Dataproc, Data Fusion, Cloud Composer, and Cloud Storage to design robust data solutions.
- Create and maintain efficient data models and schemas for analytical reporting, machine learning pipelines, and real-time processing.
- Collaborate closely with data scientists, analysts, and business stakeholders to understand requirements and convert them into technical data solutions.
- Implement best practices for data governance, security, privacy, and compliance across the entire data lifecycle.
- Monitor, debug, and optimize pipeline performance ensuring minimal latency and high throughput.
- Design and maintain APIs and microservices for data integration across platforms.
- Perform advanced data quality checks, anomaly detection, and validation to ensure data accuracy and consistency.
- Implement CI/CD for data engineering projects using GCP-native DevOps tools.
- Stay updated with emerging GCP services and industry trends to continuously improve existing solutions.
- Create detailed documentation for data processes, workflows, and standards to enable smooth knowledge transfer.
- Support the migration of on-premise data systems to GCP, ensuring zero downtime and efficient cutover.
- Automate repetitive workflows, deployment processes, and monitoring systems using Python, Shell scripting, or Terraform.
- Provide mentoring and technical guidance to junior data engineers in the team.
Required Skills & Experience
- 5+ years of experience in data engineering with a strong focus on cloud-based data solutions.
- Hands-on expertise with Google Cloud Platform (GCP) and services including BigQuery, Dataflow, Pub/Sub, Dataproc, Data Fusion, Cloud Composer, and Cloud Storage.
- Strong proficiency in SQL, including query optimization, performance tuning, and working with large datasets.
- Advanced programming skills in Python, Java, or Scala for building data pipelines.
- Experience with real-time data streaming frameworks such as Apache Kafka or Google Pub/Sub.
- Solid knowledge of ETL/ELT processes, data modeling (star/snowflake), and schema design for both batch and streaming use cases.
- Proven track record of building data lakes, warehouses, and pipelines that can scale with enterprise-level workloads.
- Experience integrating diverse data sources including APIs, relational databases, flat files, and unstructured data.
- Knowledge of Terraform, Infrastructure as Code (IaC), and automation practices in cloud environments.
- Understanding of CI/CD pipelines for data engineering workflows and integration with Git, Jenkins, or Cloud Build.
- Strong background in data governance, lineage, and cataloging tools.
- Familiarity with machine learning workflows and enabling ML pipelines using GCP services is an advantage.
- Good grasp of Linux/Unix environments and shell scripting.
- Exposure to DevOps practices and monitoring tools such as Stackdriver or Cloud Logging/Monitoring.
- Excellent problem-solving, debugging, and analytical skills with the ability to handle complex technical challenges.
- Strong communication skills with the ability to work independently in a remote-first team environment.
Nice-to-Have Skills
- Experience with multi-cloud or hybrid environments (AWS/Azure alongside GCP).
- Familiarity with data visualization platforms such as Looker, Tableau, or Power BI.
- Exposure to containerization technologies such as Docker and Kubernetes.
- Understanding of big data processing frameworks like Spark, Hadoop, or Flink.
- Prior experience in industries with high data volume such as finance, retail, healthcare, or Background :
- Bachelors or Masters degree in Computer Science, Information Technology, Data Engineering, or a related field.
- Relevant GCP certifications (e.g., Professional Data Engineer, Professional Cloud Architect) will be highly preferred.
Why Join Us
- Opportunity to work on cutting-edge cloud data projects at scale.
- Fully remote working environment with flexible schedules.
- Exposure to innovative data engineering practices and advanced GCP tools.
- Collaborative team culture that values continuous learning, innovation, and career growth.
(ref:hirist.tech)
