665 Dataproc Jobs - Page 3

Lead data engineering squad to design, build, and optimize ETL pipelines; ensures quality and performance standards. Must have one Hetero/Homogeneous migration experience Interact closely with Business Analysis team, ETL team and BI Reporting teams to ensure understanding of proper use of data architecture. Design, build, and maintain physical databases, dimensional data models, ETL layer design and data integration strategies. Deep understanding of data modelling, data warehousing, and data quality. Candidate should possess excellent problem solving abilities and data analysis Good written and verbal communication skills. Roles & Responsibilities: Design and implement effective database sol...

Job Summary: We are looking for a skilled GCP Data Engineer to design, build, and optimize data pipelines and solutions on Google Cloud Platform. The ideal candidate will have hands-on experience with BigQuery, Composer, Dataproc, Python, PySpark, Airflow, and SQL , and will play a key role in ensuring data reliability, scalability, and efficiency across our data ecosystem. Collaborate with data analysts, scientists, and other stakeholders to deliver reliable data solutions. Monitor and troubleshoot data pipelines and ensure data accuracy and consistency. Optimize cloud resources for cost efficiency and performance . Maintain clear technical documentation for all data processes and workflows...

Role & responsibilities Design, build, and maintain scalable ETL/ELT data pipelines using PySpark Develop and optimize data workflows leveraging GCP DataProc, BigQuery, Cloud Storage, and Cloud Compose Ingest, transform, and integrate structured and unstructured data from diverse sources Collaborate with Data Scientists, Analysts, and cross-functional teams to deliver reliable, real-time data solutions Ensure performance, scalability, and reliability of data platforms Implement best practices for data governance, security, and quality Must-Have Skills: Strong hands-on experience in PySpark and the Apache Spark ecosystem Proficiency in working with GCP services, especially DataProc, BigQuery,...

Overview Join the Prodapt team in building a unified, enterprise-scale solution for model, data, and service management across the AI/ML model development. Responsibilities Proficiency in Python and experience with pyspark and ML/AI model development (TensorFlow, scikit-learn, MLflow). Experience in building applications with Java, including the use of Spring for backend development. Hands-on experience with core GCP services, including GKE, Dataproc, BigQuery, and Google Cloud Storage (GCS). Strong understanding of data storage and management (Oracle Exadata, HDFS, Hive, GCS, key-value stores). Experience with CI/CD tools and automation for ML/data workflows. Knowledge of security best prac...

Your Role Excellent Communication Skills. Experience working with Fact and Dimension tables, SCD. Proven analytical skills and Problem-solving attitude. Any other GCP services or databases like Datastore, Bigtable, Spanner, Cloud Run, Cloud Functions etc. Should have worked on handling big data. experience in Agile methodologies ETL, ELT skills, Data movement skills, Data processing skills. Certification on Professional Google Cloud Data engineer will be an added advantage. Ability to effectively function in a cross-teams environment Your Profile Require 5+ years experience and relevant is minimum 3 years' experience in GCP Data Engineering. Strong experience in GCP Services, data engineerin...

Data engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets. - Grade Specific The involves leading and managing a team of data engineers, overseeing data engineering projects, ensuring technical excellence, and fostering collaboration with stakeholders. They play a critical role in driving the success of data engineering initiatives and ensuring the delivery of reliable and high-quality data solutions to support the organization's data-driven objectives. Skills (competencies) Ab Initio Agile (Software Development Framework) Apache Hadoop AW...

About The Role Project Role : Data Governance Practitioner Project Role Description : Establish and enforce data governance policies to ensure the accuracy, integrity, and security of organizational data. Collaborate with key stakeholders to define data standards, facilitate effective data collection, storage, access, and usage; and drive data stewardship initiatives for comprehensive and effective data governance. Must have skills : Snowflake Data Warehouse Good to have skills : Python (Programming Language), Google Cloud SQL, Microsoft SQL Server Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Governance Practitioner, ...

We need a GCP Big data engineer. who has experience into GCP Technologies. like GCP, Pyspark, Bigdata, Big Querry, Spark, and must have GCP, SQL, Pyspark, ETL process experience and who can handle a leadership role.

Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be skilled in ETL tools such as Informatica Glue Databricks and DataProc with coding expertise in Python PySpark and SQL. Works independently and has a deep understanding of data warehousing solutions including Snowflake BigQuery Lakehouse and Delta Lake. Capable of calculating costs and understanding performance issues related to data solutions. Outcomes: Act creatively to develop pipelines and applications by selecting appropriate technical options optimizing application development mainten...

Mandatory skill : ETL_GCP_Bigquery Develop, implement, and optimize ETL/ELT pipelines for processing large datasets efficiently. Work extensively with BigQuery for data processing, querying, and optimization. Utilize Cloud Storage, Cloud Logging, Dataproc, and Pub/Sub for data ingestion, storage, and event-driven processing. Perform performance tuning and testing of the ELT platform to ensure high efficiency and scalability. Debug technical issues, perform root cause analysis, and provide solutions for production incidents. Ensure data quality, accuracy, and integrity across data pipelines. Collaborate with cross-functional teams to define technical requirements and deliver solutions. Work i...

Should be capable of developing/configuring data pipelines in a variety of platforms and technologies Possess the following technical skills SQL, Python, Pyspark, Hive, ETL, Unix, Control-M (or similar scheduling tools) Can demonstrate strong experience in writing complex SQL queries to perform data analysis on Databases SQL server, Oracle, HIVE etc. Experience with GCP (particularly Airflow, Dataproc, Big Query) is an advantage Have experience with creating solutions which power AI/ML models and generative AI Ability to work independently on specialized assignments within the context of project deliverables Take ownership of providing solutions and tools that iteratively increase engineerin...

About the Role This role is part of the MarTech Data Engineering POD supporting a large-scale ESP (Email Service Provider) migration program from Oracle Responsys to Adobe Journey Optimizer. The initiative aims to build a scalable, real-time marketing data foundation integrating customer journeys, preference orchestration, and omnichannel personalization across email, mobile, and direct channels. The offshore team will focus on designing, building, and maintaining data pipelines across SAS , OES , Wunderkind , and AEP (Adobe Experience Platform), enabling real-time campaign execution and advanced targeting capabilities. Key Responsibilities Design and develop data ingestion, transformation, ...

Greetings from Getronics! We are looking for a AI/ML Engineer. We are seeking a solution-oriented engineer who can identify where AI/LLM capabilities can transform business operations and architect practical integrations within enterprise software systems. This is not a traditional ML engineering or data science role focused on model training and deployment infrastructure. Key Responsibilities AI Solution Strategy & Innovation • Identify opportunities where AI/LLM integration can solve business problems and add value • Evaluate emerging AI technologies (GPT, Claude, open-source LLMs, agent frameworks) for enterprise applicability • Assess and recommend appropriate AI tools and frameworks bas...

Type: Contract Description: GCPdata engineer Big Query SQL Python Talend ETL ProgrammerGCPor Any Cloud technology. Experienced in GCP data engineer Big Query SQL Python Talend ETL Programmer GCPor Any Cloud technology. Good experience in building the pipeline of GCP Components to load the data into Big Query and to cloud storage buckets. Excellent Data Analysis skills. Good written and oral communication skills Self-motivated able to work independently

Role Overview: Industrial System Analytics (ISA) is a product group that develops cloud analytic solutions using GCP tools and techniques. As a Tech Anchor/Solution Architect within the Vehicle Product Management Product Line, you will be responsible for providing technical leadership and guidance to the development team, driving the design and implementation of cloud analytic solutions using GCP tools and techniques. This role is ideal for individuals who are technically and product-oriented, experienced in managing products, building roadmaps, enhancing customer satisfaction, and interested in working with a portfolio of strategic analytic solutions. Key Responsibilities: - Provide technic...

Role & responsibilities We are seeking a GCP Solutions Expert who can bridge the gap between business vision and technical execution. This role is not for a pure developer its for someone who understands end-to-end cloud architectures, can visualize solutions, and has experience guiding teams in migrating enterprise data platforms (especially Informatica GCP). Youll work closely with leadership to design, communicate, and document scalable GCP solutions across data engineering, automation, and analytics domains. Key Responsibilities • Understand high-level business problems and design solution architectures using GCP-native services (BigQuery, Dataflow, Dataproc, Composer, Cloud Functions, D...

Overview: TekWissen is a global workforce management provider throughout India and many other countries in the world. The below client is a global company with shared ideals and a deep sense of family. From our earliest days as a pioneer of modern transportation, we have sought to make the world a better place one that benefits lives, communities and the planet Job Title: Data Engineering Engineer II Location: Chennai Work Type: Hybrid Position Description: Employees in this job function are responsible for designing, building, and maintaining data solutions including data infrastructure, pipelines, etc. for collecting, storing, processing and analyzing large volumes of data efficiently and ...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Microsoft SQL Server, Google Cloud Data ServicesMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, e...

Your role and responsibilities As an Application Developer, you will lead IBM into the future by translating system requirements into the design and development of customized systems in an agile environment. The success of IBM is in your hands as you transform vital business needs into code and drive innovation. Your work will power IBM and its clients globally, collaborating and integrating code into enterprise systems. You will have access to the latest education, tools and technology, and a limitless career path with the worlds technology leader. Come to IBM and make a global impact! IBMs Cloud Services are focused on supporting clients on their cloud journey across any platform to achiev...

Cloud Leader - CloudOps Experience : 8 to 10 years B.Sc Computers, BE, B.Tech or with equivalent experience Location : Mangalore (Onsite) Skills Required 5+ years of technical lead or technical project management experience in cloud-based environments. Proven experience leading cross-functional teams in Cloud Operations or Cloud Migration. Strong understanding of Cloud Services. Proficient with ticketing tools like Jira SM, ServiceNow, Freshservice. Excellent communication, leadership, and stakeholder management skills. Strong knowledge of ITIL process. Strong hands-on experience with GCP services: BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Composer, Dataproc, Cloud Functions. Program...

Role & responsibilities- GCP Data Engineer , Big Querry, Data flow, Data Composer, SQL, Python, Pyspark along with 3+ Years strong GCP experience & worked as Module lead / Tech Lead Exp - 4 to 15 yrs Locations - Pan India Work Mode - Hybrid Notice Period - Immediate to 30 Days & Serving notice period

Role & responsibilities- GCP Data Engineer , Big Querry, Data flow, Data Composer, SQL, Python, Pyspark along with 3+ Years strong GCP experience & worked as Module lead / Tech Lead Exp - 4 to 15 yrs Locations - Pan India Work Mode - Hybrid Notice Period - Immediate to 30 Days & Serving notice period

Role & responsibilities- GCP Data Engineer , Big Querry, Data flow, Data Composer, SQL, Python, Pyspark along with 3+ Years strong GCP experience & worked as Module lead / Tech Lead Exp - 4 to 15 yrs Locations - Pan India Work Mode - Hybrid Notice Period - Immediate to 30 Days & Serving notice period

Job Description:_ Expertise in Google Data Engineering services Big Query, Dataproc etc • Strong experience in programming languages such as Python, PySpark data manipulation and engineering tasks. • Expertise in SQL and NoSQL databases • In-depth knowledge of data warehousing solutions • Deep understanding on data warehousing concepts • Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using tools like Apache Airflow, Talend, or Informatica.

Overview GCP, Big Query, Python, Pyspark, Dataproc, Airflow, SQL Responsibilities GCP, Big Query, Python, Pyspark, Dataproc, Airflow, SQL Requirements GCP, Big Query, Python, Pyspark, Dataproc, Airflow, SQL