665 Dataproc Jobs - Page 2

Design, develop, and maintain scalable ETL/ELT data pipelines on GCP (BigQuery, Dataflow, Cloud Composer, Pub/Sub, etc) Build and optimize data models and data marts in BigQuery for analytical and reporting use cases Ensure data quality, integrity, and security across the data lifecycle Implement data transformation logic using SQL, Python, and Cloud Dataflow/Dataproc Collaborate with business and analytics teams to understand data requirements and deliver efficient solutions Automate workflows and orchestrate pipelines using Cloud Composer (Airflow) Monitor and optimize BigQuery performance and manage cost efficiency Support CI/CD deployment processes and maintain version control using Git ...

Your role and responsibilities As a Brand Technical Specialist, you'll work closely with clients to develop relationships, understand their needs, earn their trust and show them how IBM's industry leading solutions will solve their problems whilst delivering value to their business. We're committed to success. In this role, your achievements will drive your career, team, and clients to thrive. A typical day may involve: Strategic Mainframe Solutions: Crafting client strategies for mainframe infrastructure and applications. Comprehensive zStack Solutions: Defining and detailing IBM zStack solutions for client enhancement. Effective Client Education: Delivering simplified proof of concepts and...

Your role and responsibilities Skilled Multiple GCP services - GCS, BigQuery, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporti...

We are looking for a skilled professional with 5-10 years of experience to join our team in Bangalore, Hyderabad, Chennai, Pune, Noida, and Gurgaon. The ideal candidate will have expertise in GCP Hadoop ecosystem and be able to handle batch data processing on the Hadoop ecosystem. Roles and Responsibility Design and develop scalable data pipelines using GCP Hadoop ecosystem. Process large datasets using Scala Spark, Hive, and other relevant tools. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain technical documentation for data processing systems. Troubleshoot and resolve issues related to data processing and system performance. Op...

We are looking for a skilled professional with 5-10 years of experience to join our team as a GCP expert. The ideal candidate will have a strong background in BigQuery, GCP, and coding languages like Java, Python, and Hadoop. This position is available in Bangalore, Hyderabad, Chennai, Pune, Noida, and Gurgaon. Roles and Responsibility Design and develop scalable and efficient data pipelines using BigQuery and GCP. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data architectures using Cloud SQL and Apache Airflow. Ensure high-quality data processing and analysis by implementing robust testing and validation framework...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Adobe Experience Manager (AEM) UI Development, Adobe Analytics, Java Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing user interfaces, and ensuring that applications function seamlessly. You ...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Overview Design and implement complex ETL/ELT pipelines using PySpark and Airflow for large-scale data processing on GCP.Lead data migration initiatives, including automating the movement of Teradata tables to BigQuery, ensuring data accuracy and consistency. Develop robust frameworks to streamline batch and streaming data ingestion workflows, leveraging Kafka, Dataflow, and NiFi. Collaborate with data scientists to build ML-ready data layers and support analytics solutions. Conduct proof of concepts (POCs) and document performance benchmarking for data throughput and velocity, ensuring optimized data workflows. Enhance CI/CD pipelines using Jenkins and GitLab for efficient deployment and mo...

Job Summary: We are seeking an experienced ML Data Scientist to design, develop, and deploy machine learning models that solve complex business problems and deliver measurable outcomes. The ideal candidate will possess strong analytical skills, deep understanding of statistical and ML techniques, and hands-on experience in model development and deployment in production environments. Key Responsibilities: Understand business objectives and translate them into data-driven solutions . Design, develop, and implement machine learning models, algorithms, and predictive analytics . Perform data cleaning, preprocessing, feature engineering , and exploratory data analysis (EDA). Evaluate model perfor...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Overview Project Management Big Data Technologies Google Cloud Platform (GCP) API Development Agile/Scrum Risk Management Budget Management Stakeholder Management Technical Leadership Communication Responsibilities Project Planning and Execution: Define project scope, goals, and deliverables in collaboration with stakeholders. Develop detailed project plans, including timelines, resource allocation, and risk management strategies. Manage project execution, ensuring adherence to timelines and budgets. Utilize project management methodologies (e.g., Agile, Waterfall, Scrum) as appropriate. Technical Leadership: Provide technical guidance and leadership to project teams. Understand and contribu...

Overview Project Management Google Cloud Platform (GCP) Agile/Scrum Risk Management Budget Management Stakeholder Management Technical Leadership Communication Responsibilities Project Planning and Execution: Define project scope, goals, and deliverables in collaboration with stakeholders. Develop detailed project plans, including timelines, resource allocation, and risk management strategies. Manage project execution, ensuring adherence to timelines and budgets. Utilize project management methodologies (e.g., Agile, Waterfall, Scrum) as appropriate. Technical Leadership: Provide technical guidance and leadership to project teams. Understand and contribute to the technical architecture on GC...

Design, develop, test, and maintain full-stack software using Java, Spring Boot, Angular, GCP, 3DX, SAP, or Pega. Collaborate with teams, ensure quality, optimize performance, and deliver secure, scalable, cloud-based solutions.

About us: Working at Tech Holding isnt just a job, its an opportunity to be a part of something bigger We are a full-service consulting firm that was founded on the premise of delivering predictable outcomes and high-quality solutions to our clients Our founders and team members have industry experience and have held senior positions in a wide variety of companies- from emerging startups to large Fortune 50 firms- and we have taken our combined experiences and developed a unique approach that is supported by the principles of deep expertise, integrity, transparency, and dependability We are looking for talented Data Engineer to join our team As a Data Engineer, you will be responsible for de...

About us: Working at Tech Holding isnt just a job, its an opportunity to be a part of something bigger We are a full-service consulting firm that was founded on the premise of delivering predictable outcomes and high-quality solutions to our clients Our founders and team members have industry experience and have held senior positions in a wide variety of companies- from emerging startups to large Fortune 50 firms- and we have taken our combined experiences and developed a unique approach that is supported by the principles of deep expertise, integrity, transparency, and dependability We are looking for talented Data Engineer to join our team As a Data Engineer, you will be responsible for de...

Role Overview: As a Director of Cloud Architecture and Engineering at Deutsche Bank in Pune, India, you will be leading the global cloud infrastructure architecture across Corporate Bank domains. You will be responsible for creating domain-level architecture roadmaps, building solutions, and providing technical leadership to development teams. Your role will involve collaborating with various stakeholders, designing complex solutions, and ensuring successful delivery of software. Key Responsibilities: - Lead the global cloud infrastructure architecture across Corporate Bank domains - Create domain level architecture roadmaps to ensure long-term business strategy success - Build solutions and...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Microsoft SQL Server, Google Cloud Data ServicesMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, e...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Microsoft SQL Server, Google Cloud Data ServicesMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, e...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery, Microsoft SQL Server, GitHub, Google Cloud Data Services Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery, Microsoft SQL Server, GitHub, Google Cloud Data Services Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve...

About us: Working at Tech Holding isnt just a job, its an opportunity to be a part of something bigger We are a full-service consulting firm that was founded on the premise of delivering predictable outcomes and high-quality solutions to our clients Our founders and team members have industry experience and have held senior positions in a wide variety of companies- from emerging startups to large Fortune 50 firms- and we have taken our combined experiences and developed a unique approach that is supported by the principles of deep expertise, integrity, transparency, and dependability We are looking for talented Data Engineer to join our team As a Data Engineer, you will be responsible for de...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Microsoft SQL Server, Google Cloud Data ServicesMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, e...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Microsoft SQL Server, Google Cloud Data ServicesMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, e...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Microsoft SQL Server, Google Cloud Data ServicesMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, e...