Jobs
Interviews

419 Dataproc Jobs - Page 5

Setup a job Alert
JobPe aggregates results for easy application access, but you actually apply on the job portal directly.

3.0 - 8.0 years

1 - 1 Lacs

chennai

Work from Office

Overview: TekWissen is a global workforce management provider throughout India and many other countries in the world. The below clientis a global company with shared ideals and a deep sense of family. From our earliest days as a pioneer of modern transportation, we have sought to make the world a better place one that benefits lives, communities and the planet Job Title: Software Engineer III - Core Engineer III Location: Chennai Work Type: Hybrid Position Description: GCP AI Data Engineer Skills Required: TERRAFORM, Big Query, AIRFLOW, POSTGRES, Data Flow, Data Fusion, Dataproc, Cloud Storage Experience Required: 2+ years' experience Education Required: Bachelor's Degree TekWissen Group is an equal opportunity employer supporting workforce diversity.

Posted 2 weeks ago

Apply

9.0 - 14.0 years

30 - 45 Lacs

noida, bengaluru, delhi / ncr

Work from Office

8+ years of extensive experience in data engineering, data architecture, or analytics. GCP using BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Composer; expertise in SQL, Python, ETL/ELT, IaC, security, governance, and migration.

Posted 2 weeks ago

Apply

7.0 - 12.0 years

20 - 30 Lacs

hyderabad

Remote

Role & responsibilities The Lead Data Engineer will build, optimize, and maintain high-throughput data pipelines on GCP, ensuring resilient imports, freshness monitoring, and automation-readiness. This role leads engineering teams to deliver scalable data solutions. Responsibilities Develop and maintain large-scale ingestion and transformation pipelines on GCP . Implement monitoring and alerting for data freshness and resiliency . Optimize ETL/ELT using BigQuery, Dataflow, Pub/Sub, Dataproc . Collaborate with AI teams to embed automation components. Drive best practices in DevOps, CI/CD, and testing. Mentor engineers and lead Agile delivery cycles. Technical Skills Proficiency in GCP data stack (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage) . Strong programming skills in Python, SQL, PySpark, Java/Scala . Experience with Airflow/Cloud Composer for orchestration. Knowledge of CI/CD with Cloud Build, Cloud Deploy, Terraform for GCP . Hands-on with data quality frameworks and observability tools . Preferred candidate profile

Posted 2 weeks ago

Apply

4.0 - 9.0 years

8 - 12 Lacs

bengaluru

Work from Office

Role Overview: Assess client needs and convert business requirements into business intelligence (BI) solutions roadmap relating to complex issues involving long-term or multi-work streams. Analyze technical issues and questions identifying data needs and delivery mechanisms Implement data structures using best practices in data modeling, ETL/ELT processes, Spark, Scala, SQL, database, and OLAP technologies Manage overall development cycle, driving best practices and ensuring development of high quality code for common assets and framework components Develop test-driven solutions and provide technical guidance and heavily contribute to a team of high caliber Data Engineers by developing test-driven solutions and BI Applications that can be deployed quickly and in an automated fashion. Manage and execute against agile plans and set deadlines based on client, business, and technical requirements Drive resolution of technology roadblocks including code, infrastructure, build, deployment, and operations Ensure all code adheres to development & security standards About you: 4 year degree or equivalent experience 1+ years of software development experience preferably in a data engineering/Hadoop development (Hive, Spark etc.) Hands on Experience in Object Oriented or functional programming such as Scala / Java / Python Knowledge or experience with a variety of database technologies (Postgres, Cassandra, SQL Server) Knowledge with design of data integration using API and streaming technologies (Kafka) as well as ETL and other data Integration patterns Experience with cloud platforms like Google Cloud, AWS, or Azure. Hands on Experience on BigQuery will be an added advantage Good understanding of distributed storage(HDFS, Google Cloud Storage, Amazon S3) and processing(Spark, Google Dataproc, Amazon EMR or Databricks) Experience with CI/CD toolchain (Drone, Jenkins, Vela, Kubernetes) a plus Familiarity with data warehousing concepts and technologies. Maintains technical knowledge within areas of expertise Constant learner and team player who enjoys solving tech challenges with global team. Hands on experience in building complex data pipelines and flow optimizations Be able to understand the data, draw insights and make recommendations and be able to identify any data quality issues upfront Experience with test-driven development and software test automation Follow best coding practices & engineering guidelines as prescribed Strong written and verbal communication skills with the ability to present complex technical information in a clear and concise manner to variety of audiences

Posted 3 weeks ago

Apply

5.0 - 9.0 years

0 Lacs

chennai, tamil nadu

On-site

The ideal candidate should possess strong programming skills, particularly in Python, along with experience in ML frameworks like TensorFlow or PyTorch. You must have proven expertise in MLOps best practices, which includes model versioning, monitoring, automated deployment, and infrastructure automation. In-depth knowledge of Google Cloud Platform services is essential, specifically in Data Fusion, Dataproc, Dataflow for data processing, and pipeline orchestration. Experience with PostgreSQL databases and data integration tools like Qlik Replicate is also required. You should have expertise in security and privacy considerations for machine learning systems, including data encryption, access control, and compliance with regulations such as GDPR and HIPAA. Additionally, strong communication and leadership skills are necessary to effectively engage both technical and non-technical stakeholders.,

Posted 3 weeks ago

Apply

3.0 - 7.0 years

17 - 22 Lacs

bengaluru

Work from Office

Your Career Our Data & Analytics group is responsible for working with various business owners/stakeholders from Sales, Marketing, People, GCS, Infosec, Operations, and Finance to solve complex business problems which will have a direct impact on the metrics defined to showcase the progress of Palo Alto Networks. We leverage the latest technologies from the Cloud & Big Data ecosystem to improve business outcomes and create through prototyping, Proof-of-Concept projects and application development. We are looking for a Staff IT Data and ML platform Engineer. Cloud engineering and business intelligence (BI) tools. The ideal candidate will be responsible for designing, implementing, and maintaining scalable platforms and analytical solutions that support our business objectives. This role requires a strong understanding of data engineering principles, as well as the ability to collaborate with cross-functional teams to deliver high-quality data platform solutions. Your Impact Design, develop, and maintain data Infrastructure to support ETL, real time pipelines, data science and AI/ML workloads Build and maintain CICD pipelines to support various workflows. Support workflow from various sources into our data warehouse or data lake environment. Collaborate with stakeholders to gather requirements and translate business needs into technical solutions. Optimize and tune existing data pipelines for performance, reliability, and scalability. Implement data quality and governance processes to ensure data accuracy, consistency, and compliance with regulatory standards. Mentor junior members of the team and provide guidance on best practices for data engineering and BI development. Qualifications Your Experience UG/PG degree in Computer Science, Engineering, or a related field or military experience required 5 to 8 years of experience in DevOps, Data platform operations and ML workloads, with a focus on building and maintaining data and AI/ML tools. Familiarity with cloud platforms such as Google Cloud Platform (GCP), and experience with relevant services (e.g. GCP Dataflow, GCP DataProc, Biq Query, Procedures, Cloud Composer etc). Experience with Big data tools Airflow, Kafka, Grafana, Prometheus, LGTM etc. Experience with object-oriented/object function scripting languages: Python/Scala, etc Strong analytical and problem-solving skills, with the ability to analyze complex data sets and derive actionable insights. Excellent communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams. Additional Information The Team Working at a high-tech cybersecurity company within Information Technology is a once in a lifetime opportunity. Youll be joined with the brightest minds in technology, creating, building, and supporting tools that enable our global teams on the front line of defense against cyberattacks.

Posted 3 weeks ago

Apply

4.0 - 8.0 years

5 - 8 Lacs

chennai

Work from Office

Responsibilities What you'll do Engineer, test, document and manage GCP Dataproc, DataFlow and VertexAI services used in high-performance data processing pipelines and Machine Learning. Help developers optimize data processing jobs using Spark, Python, and Java. Collaborate with development teams to integrate data processing pipelines with other cloud services and applications. Utilize Terraform and Tekton for infrastructure as code (IaC) and CI/CD pipelines, ensuring efficient deployment and management. Good to have Experience with Spark for large-scale data processing. Solid understanding and experience with GitHub for version control and collaboration. Experience with Terraform for infrastructure management and Tekton for continuous integration and deployment. Experience with Apache NiFi for data flow automation. Knowledge of Apache Kafka for real-time data streaming. Familiarity with Google Cloud Pub/Sub for event-driven systems and messaging. Familiarity with Google BigQuery Mandatory Key Skills Python,Java,Google Cloud Pub/Sub,Apache Kafka,Big Query,CI/CD*,Machine Learning*,Spark*

Posted 3 weeks ago

Apply

3.0 - 5.0 years

10 - 13 Lacs

chennai

Work from Office

3+ years experienced engineer who has worked on GCP environment and its relevant tools/services. (Big Query, Data Proc, Data flow, Cloud Storage, Terraform, Tekton , Cloudrun , Cloud scheduler, Astronomer/Airflow, Pub/sub, Kafka, Cloud Spanner streaming etc) 1 or 2 + years of strong experience in Python development (Object oriented/Functional Programming, Pandas, Pyspark etc) 1 or 2 + years of strong experience in SQL Language (CTEs, Window functions, Aggregate functions etc) Mandatory Key Skills Terraform,Tekton,Cloudrun,Cloud scheduler,Astronomer,Airflow,Kafka,Cloud Spanner streaming,Python development,Pandas,Pyspark,SQL,GCP,Big Query,Data Proc,Data flow,Cloud Storage*

Posted 3 weeks ago

Apply

7.0 - 12.0 years

11 - 15 Lacs

noida

Work from Office

Primary Skill(s): Lead Data Visualization Engineer with experience in Sigma BI Experience: 7+ Years in of experience in Data Visualization with experience in Sigma BI, PowerBI, Tableau or Looker Job Summary: Lead Data Visualization Engineer with deep expertise in Sigma BI and a strong ability to craft meaningful, insight-rich visual stories for business stakeholders. This role will be instrumental in transforming raw data into intuitive dashboards and visual analytics, helping cross-functional teams make informed decisions quickly and effectively. Key Responsibilities: Lead the design, development, and deployment of Sigma BI dashboards and reports tailored for various business functions. Translate complex data sets into clear, actionable insights using advanced visualization techniques. Collaborate with business stakeholders to understand goals, KPIs, and data requirements. Build data stories that communicate key business metrics, trends, and anomalies. Serve as a subject matter expert in Sigma BI and guide junior team members on best practices. Ensure visualizations follow design standards, accessibility guidelines, and performance optimization. Partner with data engineering and analytics teams to source and structure data effectively. Conduct workshops and training sessions to enable business users in consuming and interacting with dashboards. Drive the adoption of self-service BI tools and foster a data-driven decision-making culture. Required Skills & Experience: 7+ years of hands-on experience in Business Intelligence, with at least 2 years using Sigma BI. Proven ability to build end-to-end dashboard solutions that tell a story and influence decisions. Strong understanding of data modeling, SQL, and cloud data platforms (Snowflake, BigQuery, etc.). Demonstrated experience working with business users, gathering requirements, and delivering user-friendly outputs. Proficient in data storytelling, UX design principles, and visualization best practices. Experience integrating Sigma BI with modern data stacks and APIs is a plus. Excellent communication and stakeholder management skills. Preferred Qualifications: Experience with other BI tools (such as Sigma BI, Tableau, Power BI, Looker) is a plus. Familiarity with AWS Cloud Data Ecosystems (AWS Databricks). Background in Data Analysis, Statistics, or Business Analytics. Working Hours: 2 PM 11 PM IST [~4.30 AM ET 1.30 PM ET]. Communication skills: Good Mandatory Competencies BI and Reporting Tools - BI and Reporting Tools - Power BI BI and Reporting Tools - BI and Reporting Tools - Tableau Database - Database Programming - SQL Cloud - GCP - Cloud Data Fusion, Dataproc, BigQuery, Cloud Composer, Cloud Bigtable Data Science and Machine Learning - Data Science and Machine Learning - Databricks Cloud - AWS - ECS DMS - Data Analysis Skills Beh - Communication and collaboration BI and Reporting Tools - BI and Reporting Tools - Sigma BI

Posted 3 weeks ago

Apply

3.0 - 5.0 years

10 - 15 Lacs

bengaluru

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow)

Posted 3 weeks ago

Apply

5.0 - 7.0 years

15 - 17 Lacs

chennai

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow) Mandatory Key Skills java,data flow,sql,shell scripting,agile,cloud platform,google,gen,data processing,hive,sqoop,spark,hadoop,aws,big data,javascript,ansible,docker,jenkins,linux,microsoft azure,html,git,Google Cloud Platform*

Posted 3 weeks ago

Apply

5.0 - 10.0 years

12 - 20 Lacs

bengaluru

Remote

Role & responsibilities Build and deploy data platforms, data warehouses, and big data solutions across industries (BFSI, Manufacturing, Healthcare, eCommerce, IoT, Digital Twin, etc) Integrating, transforming, and consolidating data from various structured and unstructured data systems. Expert in data ingestion, transformation, storage, and analysis, often using Azure services and migration from legacy on-premise services Essential skills include SQL, Python, R and knowledge of ETL/ELT processes and big data technologies like Apache Spark, Scala, PySpark. Maintain data integrity, resolve data-related issues, and ensure the reliability and performance of data solutions Work with stakeholders to provide real-time data analytics, monitor data pipelines, and optimize performance and scalability Strong understanding of data management fundamentals, data warehousing, and data modeling. Bigdata technologies HDFS, Spark, Hbase, Hive, Sqoop, Kafka, RabbitMQ, Flink Implement seamless data integration solutions between Azure/AWS/GCP and Snowflake platforms. Identify and resolve performance bottlenecks, optimize queries, and ensure the overall efficiency of data pipelines. Lead the development and management of data infrastructure, including tools, dashboards, queries, reports, and scripts, ensuring automation of recurring tasks while maintaining data quality and integrity Implement and maintain data security measures, ensuring compliance with industry standards and regulations. Ensure data architecture aligns with business requirements and best practices. Experience in Power BI /Tableau /Looker Management, administration, and maintenance with data streaming tools such as Kafka/Confluent Kafka, Flink Experience in Test Driven Development, building libraries, and proficiency in Pandas, NumPy, Elasticsearch, Apache Beam Familiarity with CI/CD pipelines, monitoring, and infrastructure-as-code (e. g., Terraform, CloudFormation). Proficient in query optimization, data partitioning, indexing strategies, and caching mechanisms. Ensure GDPR, SOX, and other regulatory compliance across data workflows Essential skills include SQL, Python, and knowledge of ETL/ELT processes and big data technologies like Spark. Exposure in working on Event/File/Table Formats such as Avro, Parquet, Iceberg, Delta Must have Skills (Atleast one or two) Azure Data Factory (ADF), Databricks, Synapse, Data Lake Storage, Timeseries Insights, Azure SQL Database, SQL Server, Presto, SSIS AWS data services such as S3, Glue Studio, Redshift, Athena, and EMR, Redshift , Airflow, IAM, DBT, Lambda, RDS, DynamoDB, Neo4j, Amazon Neptune GCP Big query, SQL, Composer, Dataflow, Dataform, DBT, /Python, Cloud functions, Dataproc+pyspark, Python injection, Dataflow, Cloud Storage, Pub/Sub, and Vertex AI, GKE, Cloud Functions OCI Object Storage, OCI Data Integration, Oracle Database, Oracle Analytics Cloud, Oracle Analytics Cloud (OAC), Autonomous Data Warehouse (ADW), NetSuite Analytics Warehouse (NSAW), PL/SQL, Exadata

Posted 3 weeks ago

Apply

5.0 - 10.0 years

10 - 14 Lacs

noida

Hybrid

Data Engineer (SaaS-Based). Immediate Joiners Preferred. Shift : 3 PM to 12 AM IST. Good to have : GCP Certified Data Engineer. Overview Of The Role. As a GCP Data Engineer, you'll focus on solving problems and creating value for the business by building solutions that are reliable and scalable to work with the size and scope of the company. You will be tasked with creating custom-built pipelines as well as migrating on-prem data pipelines to the GCP stack. You will be part of a team tackling intricate problems by designing and deploying reliable and scalable solutions tailored to the company's data landscape.. Required Skills: 5+ years of industry experience in software development, data engineering, business intelligence, or related field with experience in manipulating, processing, and extracting value from datasets.. Extensive experience in doing requirement discovery, analysis and data pipeline solution design.. Design, build and deploy internal applications to support our technology life cycle, collaboration and spaces, service delivery management, data and business intelligence among others.. Building Modular code for multi usable pipeline or any kind of complex Ingestion Framework used to ease the job to load the data into Datalake or Data Warehouse from multiple sources.. Work closely with analysts and business process owners to translate business requirements into technical solutions.. Coding experience in scripting and languages (Python, SQL, PySpark).. Expertise in Google Cloud Platform (GCP) technologies in the data warehousing space (BigQuery, GCP Workflows, Cloud Scheduler, Secret Manager, Batch, Cloud Logging, Cloud SDK, Google Cloud Storage, IAM).. Exposure of Google Dataproc and Dataflow.. Maintain highest levels of development practices including: technical design, solution development, systems configuration, test documentation/execution, issue identification and resolution, writing clean, modular and self-sustaining code, with repeatable quality and predictability.. Understanding CI/CD Processes using Pulumi, GitHub, Cloud Build, Cloud SDK, Docker. Experience with SAS/SQL Server/SSIS is an added advantage.. Qualifications: Bachelor's degree in Computer Science or related technical field, or equivalent practical experience.. GCP Certified Data Engineer (preferred). Excellent verbal and written communication skills with the ability to effectively advocate technical solutions to other engineering teams and business audiences..

Posted 3 weeks ago

Apply

5.0 - 10.0 years

15 - 20 Lacs

chennai

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow) Mandatory Key Skills Google Cloud Platform,GCS,Big Query,Data Flow,Java

Posted 3 weeks ago

Apply

6.0 - 8.0 years

18 - 20 Lacs

chennai

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow) Keywords Product Management,Product development,Product operations,GCS,DataProc,Big Query,Composer,Data Processing,Product Design*Mandatory Key Skills Product Management,Product development,Product operations,GCS,DataProc,Big Query,Composer,Data Processing,Product Design*

Posted 3 weeks ago

Apply

5.0 - 10.0 years

15 - 30 Lacs

chennai, coimbatore, bengaluru

Hybrid

Data Engineer (GCP) Immediate Opportunity! Location: Chennai Experience: 5+ yr Job Type: Full-time We are looking for a skilled Data Engineer to join our dynamic team. If you have expertise in GCP, Python, SQL, and Big Query and want to work on cutting-edge data projects, this role is for you! Key Responsibilities: Build and maintain scalable data pipelines using Big Query, Dataflow, Pub/Sub, and Cloud Storage . Design and optimize data models for analytics and reporting. Implement robust ETL workflows to handle diverse data sources. Collaborate with analysts, data scientists, and stakeholders to deliver data-driven solutions. Tune SQL queries and workflows for performance and cost efficiency. Ensure data quality, governance, and security across platforms. Automate workflows with Cloud Composer (Apache Airflow) . Monitor, troubleshoot, and ensure pipeline reliability. Required Skills: Strong Python programming for data processing & automation. Advanced SQL skills. Hands-on experience with GCP (BigQuery, Dataflow, Pub/Sub, Cloud Storage) . Knowledge of data modeling and warehousing best practices. Experience with Apache Airflow / Cloud Composer . Familiarity with Terraform, Git, and CI/CD pipelines (a plus). Apply Now Send your updated resume to gomathi@reveilletechnologies.com with the subject line: Application – Data Engineer (GCP) This is a high-priority opening , and interviews are being scheduled immediately! Don’t miss this opportunity to work on impactful data engineering projects.

Posted 3 weeks ago

Apply

5.0 - 8.0 years

5 - 9 Lacs

hyderabad

Work from Office

As a Data Engineer, you will design and implement data pipelines and integrations across cloud platforms to support AI and GenAI workloads. You’ll work closely with Data Scientists and ML Engineers to enable GenAI-powered applications through clean, secure, and optimized data flows. Key Responsibilities - Data Engineering Build and maintain ETL/ELT pipelines for structured, semi-structured, and unstructured data. Work with data warehouses/lakehouses (Snowflake, BigQuery, Databricks, Redshift). Develop real-time streaming pipelines using Kafka, Spark, or Flink. Ensure data quality, validation, and error handling. Prepare and manage data pipelines to feed LLMs and GenAI models. Work with vector databases (FAISS, Pinecone, Weaviate, Milvus) for RAG-based solutions. Support embedding generation and prompt engineering workflows. Collaborate with AI teams to integrate GenAI APIs and frameworks (LangChain, Hugging Face, OpenAI). Deploy pipelines on cloud services (AWS Glue, Azure Data Factory, GCP Dataflow/Dataproc). Use Airflow, Dagster, or Prefect for orchestration. Required Skills Proficiency in Python, SQL, PySpark. Hands-on experience with ETL pipelines and data modeling. Knowledge of cloud data platforms (AWS, Azure, or GCP). Experience with vector databases and GenAI frameworks Familiarity with Docker, Kubernetes, CI/CD pipelines. Preferred Skills Exposure to MLOps / LLMOps practices. Experience with semantic search, embeddings, or RAG systems. Familiarity with data security, compliance, and governance.

Posted 3 weeks ago

Apply

1.0 - 2.0 years

3 - 6 Lacs

dhule

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow)

Posted 3 weeks ago

Apply

13.0 - 17.0 years

32 - 35 Lacs

noida

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow)

Posted 3 weeks ago

Apply

7.0 - 12.0 years

15 - 27 Lacs

chennai

Hybrid

Senior Bigdata Developer (GCP - BigQuery , DataFlow, DataProc, Spanner) Very good communication skill Self starter and learner Willing to work from office on Hybrid mode.

Posted 3 weeks ago

Apply

7.0 - 12.0 years

18 - 27 Lacs

chennai

Hybrid

Role & responsibilities Strong 7+ years experience in Bigdata, GCP - BQ, , DataFlow, DataProc, Spanner Good knowledge/Exp in SQL Very good communication skill Self starter and learner Willing to work from office on Hybrid mode. Preferred candidate profile

Posted 3 weeks ago

Apply

2.0 - 5.0 years

4 - 8 Lacs

hyderabad

Work from Office

About The Role Data engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets. About The Role - Grade Specific The involves leading and managing a team of data engineers, overseeing data engineering projects, ensuring technical excellence, and fostering collaboration with stakeholders. They play a critical role in driving the success of data engineering initiatives and ensuring the delivery of reliable and high-quality data solutions to support the organization"s data-driven objectives. Skills (competencies) Ab Initio Agile (Software Development Framework) Apache Hadoop AWS Airflow AWS Athena AWS Code Pipeline AWS EFS AWS EMR AWS Redshift AWS S3 Azure ADLS Gen2 Azure Data Factory Azure Data Lake Storage Azure Databricks Azure Event Hub Azure Stream Analytics Azure Sunapse Bitbucket Change Management Client Centricity Collaboration Continuous Integration and Continuous Delivery (CI/CD) Data Architecture Patterns Data Format Analysis Data Governance Data Modeling Data Validation Data Vault Modeling Database Schema Design Decision-Making DevOps Dimensional Modeling GCP Big Table GCP BigQuery GCP Cloud Storage GCP DataFlow GCP DataProc Git Google Big Tabel Google Data Proc Greenplum HQL IBM Data Stage IBM DB2 Industry Standard Data Modeling (FSLDM) Industry Standard Data Modeling (IBM FSDM)) Influencing Informatica IICS Inmon methodology JavaScript Jenkins Kimball Linux - Redhat Negotiation Netezza NewSQL Oracle Exadata Performance Tuning Perl Platform Update Management Project Management PySpark Python R RDD Optimization SantOs SaS Scala Spark Shell Script Snowflake SPARK SPARK Code Optimization SQL Stakeholder Management Sun Solaris Synapse Talend Teradata Time Management Ubuntu Vendor Management

Posted 3 weeks ago

Apply

3.0 - 8.0 years

6 - 10 Lacs

bengaluru

Work from Office

Your Role Very good Understanding of current work and the tools and technologies being used. Comprehensive knowledge and clarity on Bigquery, ETL, GCS, Airflow/Composer, SQL, Python. Experience working with Fact and Dimension tables, SCD. Minimum 3 years" experience in GCP Data Engineering. Java/ Python/ Spark on GCP, Programming experience in any one language - either Python or Java or PySpark,SQL. GCS(Cloud Storage), Composer (Airflow) and BigQuery experience. Should have worked on handling big data. Your Profile Strong data engineering experience using Java or Python programming languages or Spark on Google Cloud. Pipeline development experience using Dataflow or Dataproc (Apache Beam etc). Any other GCP services or databases like Datastore, Bigtable, Spanner, Cloud Run, Cloud Functions etc. Proven analytical skills and Problem-solving attitude. Excellent Communication Skills. What youll love about working here You can shape yourcareerwith us. We offer a range of career paths and internal opportunities within Capgemini group. You will also get personalized career guidance from our leaders. You will get comprehensive wellness benefits including health checks, telemedicine, insurance with top-ups, elder care, partner coverage or new parent support via flexible work. You will have theopportunity to learnon one of the industry"s largest digital learning platforms, with access to 250,000+ courses and numerous certifications. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem. Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.

Posted 3 weeks ago

Apply

13.0 - 17.0 years

32 - 35 Lacs

gurugram

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow)

Posted 3 weeks ago

Apply

1.0 - 2.0 years

3 - 5 Lacs

ahmedabad

Work from Office

Google Cloud Platform o GCS, DataProc, Big Query, Data Flow Programming Languages o Java, Scripting Languages like Python, Shell Script, SQL Google Cloud Platform o GCS, DataProc, Big Query, Data Flow 5+ years of experience in IT application delivery with proven experience in agile development methodologies 1 to 2 years of experience in Google Cloud Platform (GCS, DataProc, Big Query, Composer, Data Processing like Data Flow)

Posted 3 weeks ago

Apply
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Featured Companies