647 Dataproc Jobs - Page 5

Project description We've been engaged by an Investment Banking client in the Corporate, Commercial and Institutional Banking (CCIB) potfolio to work on the projects for managing analytics support for all the key initiatives of all segments/products . Responsibilities Choosing the right technologies for our use cases, deploy and operate. Setting up Data stores structured, semi structured and non-structured. Secure data at rest via encryption Implement tool to access securely multiple data sources Implement solutions to run real-time analytics Use container technologies Skills Must have 6+ years of relevant technology experience Programming Skills: Strong proficiency in Java or Scala. Solid u...

About The Role Project Role : Data Science Practitioner Project Role Description : Formulating, design and deliver AI/ML-based decision-making frameworks and models for business outcomes. Measure and justify AI/ML based solution values. Must have skills : Google Cloud Platform Architecture Good to have skills : NA Minimum 18 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Technology Architect, you will design, develop, and deliver advanced analytics and machine learning solutions leveraging Google Clouds AI/ML capabilities. You will define models and workflows to meet performance, scalability, and accuracy requirements, ensuring busine...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Google Cloud Data Services, Microsoft SQL Server Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating d...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Microsoft SQL Server, Google Cloud Data Services Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating d...

Job Title: Google Data Engineering Architect Location: Pune, India Experience: 68 years Maximum Budget: 35 LPA About the Role We are looking for a highly skilled Data Engineering Architect to join our team in Pune. In this role, you will design, build, and scale Google’s large-scale data infrastructure. You will be a technical leader, driving innovation, ensuring data quality, and mentoring a team of engineers. This is a high-impact role ideal for someone passionate about big data, cloud technologies, and architecting complex data systems . Key Responsibilities Design & Architecture: Design and implement scalable, reliable, and efficient data architectures for Google products and services. D...

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using Dataproc, Data Flow, Bigtable, and other GCP services. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs. Develop automated testing frameworks to ensure high-quality code delivery. Troubleshoot complex issues related to big data processing and storage on Google Cloud Platforms. Participate in code reviews to improve overall quality of the software development process. SKill Required: GCP, Data engineer Location - Chennai Preferred Work Mode: 4 days work from office Permanent Parent Payroll with V2 soft (client ford motors) ...

Who We Are Wayfair is moving the world so that anyone can live in a home they love a journey enabled by more than 3,000 Wayfair engineers and a data-centric culture Wayfairs Advertising business is rapidly expanding, adding hundreds of millions of dollars in profits to Wayfair We are building Sponsored Products, Display & Video Ad offerings that cater to a variety of Advertiser goals while showing highly relevant and engaging Ads to millions of customers We are evolving our Ads Platform to empower advertisers across all sophistication levels to grow their business on Wayfair at a strong, positive ROI and are leveraging state of the art Machine Learning techniques, The Advertising Optimizatio...

Job Description: We are seeking a skilled and proactive GCP Data Engineer with strong experience in Python and SQL to build and manage scalable data pipelines on Google Cloud Platform (GCP) . The ideal candidate will work closely with data analysts, architects, and business teams to enable data-driven decision-making. Key Responsibilities: Design and develop robust data pipelines and ETL/ELT processes using GCP services Write efficient Python scripts for data processing, transformation, and automation Develop complex SQL queries for data extraction, aggregation, and analysis Work with tools like BigQuery, Cloud Storage, Cloud Functions , and Pub/Sub Ensure high data quality, integrity, and g...

As a Senior Engineer in the Data - IT squad at Deutsche Bank located in Pune, India, you will play a crucial role in designing solutions aligned with the bank's architecture principles and strategic goals. Your primary responsibility will involve collaborating with cloud engineers to develop cutting-edge data ingestion and processing solutions using Google Cloud Platform (GCP) capabilities. - Understand and serve as a Subject Matter Expert (SME) for the current systems landscape in Private Bank/Wealth Management, including end-to-end business process flows. - Document business and technology requirements for change projects, collaborating with technical resources to provide architectural and...

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using Dataproc, Data Flow, Bigtable, and other GCP services. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs. Develop automated testing frameworks to ensure high-quality code delivery. Troubleshoot complex issues related to big data processing and storage on Google Cloud Platforms. Participate in code reviews to improve overall quality of the software development process. Work Mode: 4 days work from office Permanent Parent Payroll with V2 soft (client ford motors) POC: Kavinila Interested Candidate can send your updated resume ...

skilled and motivated Data Engineer with hands-on experience in cloud platforms like AWS, Azure, or GCP to join our growing data team. have strong knowledge of data architecture, pipeline development, and cloud-based data processing.

Dear Candidate Greetings from TATA Consultancy Services Job Openings at TCS Skill : GCP Data Engineer (GCP Services (DataProc, Big Query, Composer) Good knowledge of Spark (Scala and PySpark)) Exp range: 6 yrs to 10 yrs Location: Pune Notice period Immediate Pls find the Job Description below. Data Engineer will be responsible for providing expertise in implementation of data ingestion pipeline using GCP native managed services. Strong experience with relational and non-relational databases in Cloud with billions of records (structured & unstructured data) Ability to design & develop data flow pipelines from scratch Excellent problem solving & debugging skills along with designing skills Mus...

About Happiest Minds:** Happiest Minds is a leading digital transformation and technology services company, empowered by the mission of enhancing the happiness of our customers, employees, and society at large. We provide cutting-edge solutions leveraging advanced technologies to drive meaningful outcomes for our clients. Location: Manyata Tech Park. JD: Need someone with GCP + Looker / LookML experience Strong knowledge of data processing in GCP , Spark Deep understanding of Big Query architecture, best practices, and performance optimization. Experience with DataProc for running Hadoop/ Spark jobs on GCP. Proficiency in LookML for building data models and metrics. Knowledge of building rep...

Role Overview: As an experienced architect with over 15 years of overall experience and 6+ years in building cloud applications in production, your role will involve leading global cloud infrastructure architecture in Corporate Bank Technology. You will be responsible for creating domain level architecture roadmaps, ensuring long-term business strategy success, reducing complexity, and managing costs effectively. Additionally, you will build solutions and deliver to production as needed for TAS business, playing a critical role in designing blueprints for GenAI and cloud infrastructure and migration projects. Key Responsibilities: - Lead global cloud infrastructure architecture across domain...

As an Associate Software Developer.. As an entry level Application Developer at IBM, you'll work with clients to co-create solutions to major real-world challenges by using best practice technologies, tools, techniques, and products to translate system requirements into the design and development of customized systems. In your role, you may be responsible for: Working across the entire system architecture to design, develop, and support high quality, scalable products and interfaces for our clients Collaborate with cross-functional teams to understand requirements and define technical specifications for generative AI projects Employing IBM's Design Thinking to create products that provide a ...

Design, build, and manage scalable cloud-native apps on GCP using services like Compute Engine, BigQuery, Cloud Functions, and Kubernetes. Develop infrastructure with Terraform. Deploy, test, integrate apps while ensuring reliability and performance.

GCP DE Responsibilities Work closely with our data science team to help build complex algorithms that provide unique insights into our data Use agile software development processes to make iterative improvements to our back-end systems Model front-end and back-end data sources to help draw a more comprehensive picture of user flows throughout the system and to enable powerful data analysis Build data pipelines that clean, transform, and aggregate data from disparate sources Develop models that can be used to make predictions and answer questions for the overall business Required skills and qualifications 7 or more years of experience with Python, SQL, and data visualization/exploration tools...

Key Responsibilities: Develop Python-based solutions using Git/Poetry for code and library management. Work with data processing libraries such as Pandas, Polars, and others. Deploy and manage data pipelines on Databricks and Azure Blob Storage. Monitor and troubleshoot pipelines, logs, and cloud resources. Required Skills & Qualifications: Proficiency in Python , including experience with Git/Poetry for code and dependency management. Expertise in data processing libraries such as Pandas, PySpark, Dataframes, Polars, or equivalent. Familiarity with Azure cloud services , Databricks , and Azure Blob Storage . Ability to read and analyze logs for debugging and performance monitoring . Strong ...

Data Engineer Join CustomerLabs! "Yesterday is history, tomorrow is a mystery, but today is a gift." Master Oogway At CustomerLabs , were building the future of digital marketing with the power of data. As a Data Engineer , youll design and manage data pipelines, warehouses, and analytics platforms that transform raw marketing data into powerful insights. Youll work with cross-functional teams (Data Scientists, Product Managers, Martech specialists) and build scalable solutions for real-time marketing analytics. What You’ll Do Build & maintain enterprise-grade ETL/ELT pipelines Design data warehouses & lakes (BigQuery, GCP stack) Ensure data quality, security & compliance (GDPR, CCPA) Collab...

Greetings from Getronics! We have permanent opportunities for GCP Data Engineers in Chennai Location . Hope you are doing well! This is Abirami from Getronics Talent Acquisition team. We have multiple opportunities for GCP Data Engineers for our automotive client in Chennai Sholinganallur location. Please find below the company profile and Job Description. If interested, please share your updated resume, recent professional photograph and Aadhaar proof at the earliest to abirami.rsk@getronics.com. Company : Getronics (Permanent role) Client : Automobile Industry Experience Required : 5+ Years in IT and minimum 3+ years in GCP Data Engineering Location : Chennai (Elcot - Sholinganallur) Work ...

About The Role Project Role : Data Architect Project Role Description : Define the data requirements and structure for the application. Model and design the application data structure, storage and integration. Must have skills : Google Cloud Platform Architecture Good to have skills : NA Minimum 15 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Architect, you will design and deliver end-to-end data architecture solutions for platforms, products, or engagements on Google Cloud. You will define architectures that meet performance, scalability, security, and compliance requirements while ensuring data integrity and accessibility. Yo...

Position Description: Leverage modern frameworks, open source tools, and cloud technologies to develop software effectively through paired programming and other methodologies like test-driven development, continuous integration and deployment. Develop working, tested code across the full stack of technologies. Build distributed, service-oriented, cloud micro service-based architectures. Develop components across all tiers of the application stack, integrate and deploy developed software, and modify CI/CD pipeline and scripts as vital to improve continuous integration practices. Identify minimal viable product and decompose features by story slicing. Identify inter-team dependencies, and ensu...

As an experienced GCP Administrator, your role will involve managing and configuring roles/permissions in GCP IAM, optimizing Big Query service, collaborating with various teams for efficient data management in GCP, creating automations and monitoring mechanisms, designing GCP-specific cloud architecture, provisioning and de-provisioning GCP accounts, and keeping technical documentation up to date. You will also be required to stay up to date on GCP announcements and services. Key Responsibilities: - Manage and configure roles/permissions in GCP IAM following the principle of least privileged access - Optimize Big Query service by managing slot assignments and SQL Queries, adopting FinOps pr...

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow)

Responsibilities What you'll do Engineer, test, document and manage GCP Dataproc, DataFlow and VertexAI services used in high-performance data processing pipelines and Machine Learning. Help developers optimize data processing jobs using Spark, Python, and Java. Collaborate with development teams to integrate data processing pipelines with other cloud services and applications. Utilize Terraform and Tekton for infrastructure as code (IaC) and CI/CD pipelines, ensuring efficient deployment and management. Good to have Experience with Spark for large-scale data processing. Solid understanding and experience with GitHub for version control and collaboration. Experience with Terraform for infras...