647 Dataproc Jobs - Page 4

As a lead engineer within the Transaction Monitoring department, you will lead and drive forward critical engineering initiatives and improvements to our application landscape whilst supporting and leading the engineering teams to excel in their roles. You will be closely aligned to the architecture function and delivery leads, ensuring alignment with planning and correct design and architecture governance is followed for all implementation work. You will lead by example and drive and contribute to automation and innovation initiatives with the engineering teams. Join the fight against financial crime with us! Your key responsibilities Experienced hands-on cloud and on-premise engineer, lead...

Role Overview: You will be a Senior Engineer, VP based in Pune, India. Your role will involve managing or performing work across multiple areas of the bank's IT Platform/Infrastructure, including analysis, development, and administration. You may also be responsible for providing functional oversight of engineering delivery for specific departments. Your key responsibilities will include planning and developing engineering solutions to achieve business goals, ensuring reliability and resiliency in solutions, promoting maintainability and reusability, reviewing engineering plans and quality, and participating in industry forums to drive the adoption of innovative technologies in the bank. You...

As a Technical Architect specializing in Google Cloud Platform (GCP) Data Engineering at Atos, your role involves designing, implementing, and optimizing scalable data solutions utilizing GCP services. Your extensive experience with GCP services, data architecture, and cloud integration ensures high performance, security, and reliability. Key Responsibilities: - Architect and optimize data solutions using GCP services such as BigQuery, Dataflow, Pub/Sub, and Cloud Storage. - Design and implement GCP-based data architectures to meet business requirements. - Develop and optimize data pipelines using GCP services such as BigQuery, Dataflow, Pub/Sub, and Cloud Storage. - Establish best practices...

Project Role : AI / ML Engineer Project Role Description : Develops applications and systems that utilize AI to improve performance and efficiency, including but not limited to deep learning, neural networks, chatbots, natural language processing. Must have skills : Google Cloud Machine Learning Services Good to have skills : GCP Dataflow, Google Dataproc, Google Pub/SubMinimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an AI / ML Engineer, you will engage in the development of applications and systems that leverage artificial intelligence to enhance performance and efficiency. Your typical day will involve collaborating with cros...

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way youd like, where youll be supported and inspired bya collaborative community of colleagues around the world, and where youll be able to reimagine whats possible. Join us and help the worlds leading organizationsunlock the value of technology and build a more sustainable, more inclusive world. Job Role Very good Understanding of current work and the tools and technologies being used. Comprehensive knowledge and clarity on Bigquery, ETL, GCS, Airflow/Composer, SQL, Python. Experience working with Fact and Dimension tables, SCD. Minimum 3 years" experience in GCP Data Engineering. Java/ Pytho...

Need to hire GCP enabled Module Leads and Leads with proficiency on data engineering technologies and languages. These folks should be able to to drive and lead migration from On Prem to GCP for Amex use cases Roles and Responsibilities 7-10 years of experience in the role of implementation of high end software products. Provides technical leadership in Big Data space (Hadoop Stack like Spark, M/R, HDFS, Hive, HBase etc) and contributes to open source Big Data technologies. Must have : Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer servics) Should be aware with columnar database e.g...

Skilled Multiple GCP services - GCS, BigQuery, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love collaborative environments that use agile...

Work Location: Pan India Experience: 5-12yrs Required Skills: Proven experience as a Data Engineer with expertise in GCP. Strong understanding of data warehousing concepts and ETL processes. Experience with BigQuery, Dataflow, and other GCP data services Design, develop, and maintain data pipelines on GCP. Implement data storage solutions and optimize data processing workflows. Ensure data quality and integrity throughout the data lifecycle. Collaborate with data scientists and analysts to understand data requirements. Monitor and maintain the health of the data infrastructure. Troubleshoot and resolve data-related issues. Thanks & Regards Suganya R Suganya@spstaffing.in

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. You Lead the Way. Weve Got Your Back. With the right backing, people and businesses have the power to progress in incredible ways. When you join Team Amex, you bec...

Job Description Senior Consultant Delivery ( Data Engineer) Architect and implement scalable, secure, and cost-efficient data pipelines leveraging GCP data products (BigQuery, Dataflow, Composer, Dataproc, Cloud Storage, etc,), Translate business requirements into technical designs for datamarts, data models, and analytics solutions, Oversee development standards, coding guidelines, and CI/CD practices to ensure code quality and maintainability, Drive adoption of data governance, lineage, and security policies across the engineering team, Collaborate closely with Product Owners, Data Architects, and stakeholders to align deliverables with business goals, Stay updated with the latest GCP adva...

Hiring Data Solutions Architect–GCP with 8+ yrs in data engineering/architecture, 5+ yrs on GCP (BigQuery, Dataflow, Pub/Sub, Dataproc, Looker). Strong SQL, Python, ETL, data modeling, Terraform, Kubernetes, CI/CD. Vertex AI knowledge a plus.

Technical Lead-Data Engg Job locations: -All Birlasoft Location. Budget -21 23 LPA Notice Period - Immediate Joiner/15 days Data Engineer Lead 8 to 10 yearsData Engineer Sr. Consultant 6 to 8yearsLocation- All Birlasoft Locations. About The Role ::Relevant experience in GCP 3+ years Strong experience in data integration with GCP using REST APIs, SQL (preferred GoogleBigQuery) , Python, GCP (Dataform, Workflow, Dataflow, Cloud Storage, Cloud Data Fusion) Familiarity in developing applications on GCP infrastructure Good to have :Integration of Google My Business, Google Ads & Google Analytics withBigQuery to implement solutions for solving client business problems. Design, develop and deploy d...

Job Description: Job Title - GCP - Senior Engineer - PD Location - Pune Role Description Our team is part of the area Technology, Data, and Innovation (TDI) Private Bank. Within TDI, Partner data is the central client reference data system in Germany. As a core banking system, many banking processes and applications are integrated and communicate via >2k interfaces. From a technical perspective, we focus on mainframe but also build solutions on premise cloud, restful services, and an angular frontend. Next to the maintenance and the implementation of new CTB requirements, the content focus also lies on the regulatory and tax topics surrounding a partner/ client. We are looking for a very mot...

About The Role : Job Title Cloud Data and AI Engineer, AVP LocationBangalore, India Role Description We are seeking an experienced Cloud (GCP) Data and AI engineer to play a key role in development and implementation of AI and Data engineering solutions. Candidate will work as an Engineering Team member in collaboration with business function and will be responsible forbuilding and deploying robust and scalable AI and data engineering solutions, in-line with banks AI strategy, driven by business value to improve operational efficiency and enhance customer experience. Deutsche Banks Corporate Bank division is a leading provider of cash management, trade finance and securities finance. We comp...

We have developed API gateway aggregators using frameworks like Hystrix and spring-cloud-gateway for circuit breaking and parallel processing. Our serving microservices handle more than 15K RPS on normal days and during saledays this can go to 30K RPS. Being a consumer app, these systems have SLAs of ~10ms Our distributed scheduler tracks more than 50 million shipments periodically fromdifferent partners and does async processing involving RDBMS. We use an in-house video streaming platform to support a wide variety of devices and networks. What Youll Do Design and implement scalable and fault-tolerant data pipelines (batch and streaming) using frameworks like Apache Spark , Flink , and Kafka...

We''re seeking a talented Data Engineer with hands-on experience in the Google Cloud data ecosystem and a proven track record of working with Vector Database, Knowledge Graphs like Neo4j and StarDog. You''ll be instrumental in designing, building, and maintaining our data infrastructure and pipelines, enabling critical insights and supporting data-driven initiatives across the organization. Responsibilities Data Pipeline Development: Design, build, and optimize robust and scalable data pipelines to ingest, transform, and load data from various sources into our data warehouse and knowledge graphs. Cloud Data Stack Expertise: Implement and manage data solutions using Google Cloud Platform (GCP...

We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms , building ETL pipelines ,and collaborating with cross-functional teams to ensure data availability,performance, and quality. Qualification:BE / B. Tech Location:Pune, Key Responsibilities Design, develop, and optimize ETL/ELT pipelines using PySpark. Ingest, clean, transform, and process structured/semi-structured/unstructured data. Work with large-scale datasets on distributed computing platforms (Hadoop, Spark, Databricks). Integrate data from multiple sources including Delta Lake, Data Lake, RDBMS, APIs. Ensure high-performance, scalabili...

* Drive the end-to-end delivery of complex projects on GCP, ensuring on-time and within-budget completion. * Provide technical leadership and guidance to project teams, ensuring adherence to architectural standards and best practices for data pipelines, data warehouses, and APIs. * Collaborate with data scientists, engineers, and architects to design and implement scalable and reliable data solutions using GCP services such as BigQuery, Dataflow, and Dataproc. * Proactively communicate project status, risks, and issues to stakeholders, managing expectations and ensuring alignment on project goals. * Establish and manage project budgets, tracking expenses and reporting on financial performanc...

As an Application Developer, you will lead IBM into the future by translating system requirements into the design and development of customized systems in an agile environment. The success of IBM is in your hands as you transform vital business needs into code and drive innovation. Your work will power IBM and its clients globally, collaborating and integrating code into enterprise systems. You will have access to the latest education, tools and technology, and a limitless career path with the world’s technology leader. Come to IBM and make a global impact! IBM’s Cloud Services are focused on supporting clients on their cloud journey across any platform to achieve their business goals. It en...

As an Associate Software Developer.. As an entry level Application Developer at IBM, you'll work with clients to co-create solutions to major real-world challenges by using best practice technologies, tools, techniques, and products to translate system requirements into the design and development of customized systems. In your role, you may be responsible for: Working across the entire system architecture to design, develop, and support high quality, scalable products and interfaces for our clients Collaborate with cross-functional teams to understand requirements and define technical specifications for generative AI projects Employing IBM's Design Thinking to create products that provide a ...

Job Title: GCP Data Engineer Work Mode: Onsite Base Location: Bangalore Experience Required: 3+ Years Employment Type: Direct Payroll with Client Job Summary: We are seeking a skilled GCP Data Engineer with strong expertise in Python and hands-on experience in designing and developing scalable data pipelines on Google Cloud Platform (GCP) . The ideal candidate should have a solid understanding of BigQuery , ETL/ELT processes , and data modeling , along with exposure to core GCP data services such as Dataflow , Pub/Sub , and Dataproc . Key Responsibilities: Design, develop, and maintain scalable data pipelines using Python, BigQuery, and GCP Dataflow. Implement efficient data models, partitio...

As a Software Engineer Practitioner at TekWissen in Chennai, you will be a crucial part of the team responsible for developing and maintaining the Enterprise Data Platform. Your main focus will be on designing, constructing, and enhancing scalable data pipelines within the Google Cloud Platform (GCP) ecosystem. By leveraging GCP Native technologies such as BigQuery, Dataform, Dataflow, and Pub/Sub, you will ensure data governance, security, and optimal performance. This role provides you with the opportunity to apply your full-stack expertise, collaborate with talented teams, and establish best practices for data engineering at the client. Key Responsibilities: - Design, build, and optimize ...

You will be a Staff, Software Engineer based in Bangalore with 10-15 years of experience. The Enterprise Business Services team at IDC Bangalore and Chennai is dedicated to data-driven, highly optimized products and implementations in Finance, Procurement, Human Resources, and Retail business functions for multiple Walmart markets using open stack technologies and ERP tools. - **Architectural Leadership:** - Lead the adoption of modern architectural patterns for application modernization. - Analyze gaps between current systems and emerging business needs to deliver cutting-edge, scalable designs. - **Drive Technical Excellence:** - Provide technical leadership in conducting impact analyses a...

About The Role Data engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets. About The Role - Grade Specific The role support the team in building and maintaining data infrastructure and systems within an organization. Skills (competencies) Ab Initio Agile (Software Development Framework) Apache Hadoop AWS Airflow AWS Athena AWS Code Pipeline AWS EFS AWS EMR AWS Redshift AWS S3 Azure ADLS Gen2 Azure Data Factory Azure Data Lake Storage Azure Databricks Azure Event Hub Azure Stream Analytics Azure Sunapse Bitbucket Change Management Client Ce...

Responsibilities Work closely with our data science team to help build complex algorithms that provide unique insights into our data Use agile software development processes to make iterative improvements to our back-end systems Model front-end and back-end data sources to help draw a more comprehensive picture of user flows throughout the system and to enable powerful data analysis Build data pipelines that clean, transform, and aggregate data from disparate sources Develop models that can be used to make predictions and answer questions for the overall business Required skills and qualifications 7 or more years of experience with Python, SQL, and data visualization/exploration tools Famili...