GCP Data Engineer

Bengaluru east

Posted:12 hours ago

Job Description

  • GCP BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, batch and real-time streaming, Databricks , Pyspark 2. Python, strong SQL Databricks with Delta Lake and Structured Streaming Responsibilities and JD in brief along with additional criteria to be considered:
  • Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms.
  • Build batch and real-time pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark and Databricks.
  • Develop and optimize Databricks, Delta Lake, Spark and Structured Streaming workloads for large-scale processing.
  • Implement high-throughput streaming solutions using Pub/Sub and Dataflow, including windowing, watermarking, late-arriving data, duplicate handling and dead-letter queues.
  • Handle high-volume TB/PB-scale data with focus on performance, scalability, reliability and cost optimization.
  • Develop complex SQL transformations, data models and analytics-ready BigQuery datasets using partitioning and clustering.
  • Orchestrate pipelines using Cloud Composer / Airflow and implement CI/CD and Terraform-based Infrastructure as Code.
  • Ensure data quality, governance, security, monitoring, alerting and production support.
  • Collaborate with architects, stakeholders and cross-functional teams in an Agile delivery model.
  • Troubleshoot and optimize end-to-end GCP, Databricks and streaming data pipelines.
Responsibilities and JD in brief along with additional criteria to be considered:
  • Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms.
  • Build batch and real-time pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark and Databricks.
  • Develop and optimize Databricks, Delta Lake, Spark and Structured Streaming workloads for large-scale processing.
  • Implement high-throughput streaming solutions using Pub/Sub and Dataflow, including windowing, watermarking, late-arriving data, duplicate handling and dead-letter queues.
  • Handle high-volume TB/PB-scale data with focus on performance, scalability, reliability and cost optimization.
  • Develop complex SQL transformations, data models and analytics-ready BigQuery datasets using partitioning and clustering.
  • Orchestrate pipelines using Cloud Composer / Airflow and implement CI/CD and Terraform-based Infrastructure as Code.
  • Ensure data quality, governance, security, monitoring, alerting and production support.
  • Collaborate with architects, stakeholders and cross-functional teams in an Agile delivery model.
  • Troubleshoot and optimize end-to-end GCP, Databricks and streaming data pipelines. Infosys is a global leader in next-generation digital services and consulting. Within the Data & Analytics unit, you will work on cutting-edge data engineering and analytics initiatives for global clients across industries. The role offers opportunities to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and large-scale enterprise data modernization programs. Employees benefit from continuous learning programs, certifications, and career growth opportunities within Infosys' data and analytics practice

Skills Required

GCP Cloud Storage Pyspark Python SQL Data Engineering Cloud Native Spark Cost Optimization Analytics Ci/cd Terraform

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Infosys logo
Infosys

IT Services and IT Consulting

Bangalore Karnataka

RecommendedJobs for You

Hinduja Global Solutions (HGS) logo

GCP Data Engineer

Hinduja Global Solutions (HGS)

Indore, Hyderabad

DataPattern logo

Bengaluru

G

GCP Data Engineer

Grow Trans IT Solutions

Chennai

G

GCP Data Engineer

Grow Trans IT Solutions

Pune