419 Dataproc Jobs - Page 17

We need someone who 6+ yrs exp and has hands on experience in migrating Google Analytics UA360 data to BigQuery. Experience working with Google Cloud data products (CloudSQL, Spanner, Cloud Storage, Pub/Sub, Dataflow, Dataproc, Bigtable, BigQuery, Dataprep, Composer, etc) Experience with IoT architectures and building real-time data streaming pipelines Experience operationalizing machine learning models on large datasets Demonstrated leadership and self-direction -- a willingness to teach others and learn new techniques Demonstrated skills in selecting the right statistical tools given a data analysis problem Understanding of Chaos Engineering Understanding of PCI, SOC2, and HIPAA compliance standards Understanding of the principle of least privilege and security best practices Experience working with Google Support. Understanding of cryptocurrency and blockchain technology

We have developed API gateway aggregators using frameworks like Hystrix and spring-cloud-gateway for circuit breaking and parallel processing. Our serving microservices handle more than 15K RPS on normal days and during saledays this can go to 30K RPS. Being a consumer app, these systems have SLAs of ~10ms Our distributed scheduler tracks more than 50 million shipments periodically fromdifferent partners and does async processing involving RDBMS. We use an in-house video streaming platform to support a wide variety of devices and networks. What Youll Do Design and implement scalable and fault-tolerant data pipelines (batch and streaming) using frameworks like Apache Spark , Flink , and Kafka . Lead the design and development of data platforms and reusable frameworks that serve multiple teams and use cases. Build and optimize data models and schemas to support large-scale operational and analytical workloads. Deeply understand Apache Spark internals and be capable of modifying or extending the open-source Spark codebase as needed. Develop streaming solutions using tools like Apache Flink , Spark Structured Streaming . Drive initiatives that abstract infrastructure complexity , enabling ML, analytics, and product teams to build faster on the platform. Champion a platform-building mindset focused on reusability , extensibility , and developer self-service . Ensure data quality, consistency, and governance through validation frameworks, observability tooling, and access controls. Optimize infrastructure for cost, latency, performance , and scalability in modern cloud-native environments . Mentor and guide junior engineers , contribute to architecture reviews, and uphold high engineering standards. Collaborate cross-functionally with product, ML, and data teams to align technical solutions with business needs. What Were Looking For 5-8 years of professional experience in software/data engineering with a focus on distributed data systems . Strong programming skills in Java , Scala , or Python , and expertise in SQL . At least 2 years of hands-on experience with big data systems including Apache Kafka , Apache Spark/EMR/Dataproc , Hive , Delta Lake , Presto/Trino , Airflow , and data lineage tools (e.g., Datahb,Marquez, OpenLineage). Experience implementing and tuning Spark/Delta Lake/Presto at terabyte-scale or beyond. Strong understanding of Apache Spark internals (Catalyst, Tungsten, shuffle, etc.) with experience customizing or contributing to open-source code. Familiarity and worked with modern open-source and cloud-native data stack components such as: Apache Iceberg , Hudi , or Delta Lake Trino/Presto , DuckDB , or ClickHouse,Pinot ,Druid Airflow , Dagster , or Prefect DBT , Great Expectations , DataHub , or OpenMetadata Kubernetes , Terraform , Docker Strong analytical and problem-solving skills , with the ability to debug complex issues in large-scale systems. Exposure to data security, privacy, observability , and compliance frameworks is a plus. Good to Have Contributions to open-source projects in the big data ecosystem (e.g., Spark, Kafka, Hive, Airflow) Hands-on data modeling experience and exposure to end-to-end data pipeline development Familiarity with OLAP data cubes and BI/reporting tools such as Tableau, Power BI, Superset, or Looker Working knowledge of tools and technologies like ELK Stack (Elasticsearch, Logstash, Kibana) , Redis , and MySQL Exposure to backend technologies including RxJava , Spring Boot , and Microservices architecture

Skills Required: Full stack + Data Engineer Position Spring Boot, Angular, Cloud Computing Skills Preferred: Google Cloud Platform , Data Flow, Dataproc, Data Fusion, , Tekton,Cloud SQL, AIRFLOW, POSTGRES, Airflow PySpark, Python, API Experience Required: 5+ years of overall experience with proficiency in Java, angular or any javascript technology with experience in designing and deploying cloud-based data pipelines and microservices using GCP tools like , Dataflow, and Dataproc. Ability to leverage best in-class data platform technologies (Apache Beam, Kafka, ) to deliver platform features, and design & orchestrate platform services to deliver data platform capabilities. Service-Oriented Architecture and Microservices: Strong understanding of SOA, microservices, and their application within a cloud data platform context. Develop robust, scalable services using Java Spring Boot, Python, Angular, and GCP technologies. Full-Stack Development: Knowledge of front-end and back-end technologies, enabling collaboration on data access and visualization layers (e.g., React, Node.js). Design and develop RESTful APIs for seamless integration across platform services. Implement robust unit and functional tests to maintain high standards of test coverage and quality. Database Management: Experience with relational (e.g., PostgreSQL, MySQL) and NoSQL databases, as well as columnar databases like BigQuery. Data Governance and Security: Understanding of data governance frameworks and implementing RBAC, encryption, and data masking in cloud environments. CI/CD and Automation: Familiarity with CI/CD pipelines, Infrastructure as Code (IaC) tools like Terraform, and automation frameworks. Manage code changes with GitHub and troubleshoot and resolve application defects efficiently. Ensure adherence to SDLC best practices, independently managing feature design, coding, testing, and production releases. Problem-Solving: Strong analytical skills with the ability to troubleshoot complex data platform and microservices issues.

Job Title: Data Engineering Engineer II Location: Chennai Work Type: Hybrid Overview: TekWissen is a global workforce management provider throughout India and many other countries in the world. The below clientis a global company with shared ideals and a deep sense of family. From our earliest days as a pioneer of modern transportation, we have sought to make the world a better place one that benefits lives, communities and the planet Position Description: Employees in this job function are responsible for designing, building, and maintaining data solutions including data infrastructure, pipelines, etc. for collecting, storing, processing and analyzing large volumes of data efficiently and accurately Key Responsibilities: Collaborate with business and technology stakeholders to understand current and future data requirements Design, build and maintain reliable, efficient and scalable data infrastructure for data collection, storage, transformation, and analysis Plan, design, build and maintain scalable data solutions including data pipelines, data models, and applications for efficient and reliable data workflow Design, implement and maintain existing and future data platforms like data warehouses, data lakes, data lakehouse etc. for structured and unstructured data Design and develop analytical tools, algorithms, and programs to support data engineering activities like writing scripts and automating tasks Ensure optimum performance and identify improvement opportunities Skills Required: GCP, Big Query,Data Flow, Dataproc, Data Fusion Experience Required: Engineer II Exp: 4+ years Data Engineering work experience Education Required: Bachelor's Degree TekWissen Group is an equal opportunity employer supporting workforce diversity.

Role: Cloud Data Engineer Location: Sholinganallur, Chennai Fulltime Skills Required: Python, dataflow, Dataproc, GCP Cloud Run, DataForm, Agile Software Development, Big Query, TERRAFORM, Data Fusion, Cloud SQL, GCP, KAFKA Skills Preferred: Java Experience Required: 8+ years Position Description: Bachelor's Degree 2+Years in GCP Services - Biq Query, Data Flow, Dataproc, DataPlex, DataFusion, Terraform, Tekton, Cloud SQL, Redis Memory, Airflow, Cloud Storage 2+ Years in Data Transfer Utilities 2+ Years in Git / any other version control tool 2+ Years in Confluent Kafka 1+ Years of Experience in API Development 2+ Years in Agile Framework 4+ years of strong experience in python, Pyspark development. 4+ years of shell scripting to develop the adhoc jobs for data importing/exporting Education Required: Bachelor's Degree

Skilled Multiple GCP services - GCS, BigQuery, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love collaborative environments that use agile methodologies to encourage creative design thinking and find innovative ways to develop with cutting edge technologies Ambitious individual who can work under their own direction towards agreed targets/goals and with creative approach to work Preferred technical and professional experience Create up to 3 bullets maxitive individual with an ability to manage change and proven time management Proven interpersonal skills while contributing to team effort by accomplishing related results as needed Up-to-date technical knowledge by attending educational workshops, reviewing publications (encouraging then to focus on required skills)

Skilled Multiple GCP services - GCS, BigQuery, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love collaborative environments that use agile methodologies to encourage creative design thinking and find innovative ways to develop with cutting edge technologies Ambitious individual who can work under their own direction towards agreed targets/goals and with creative approach to work Preferred technical and professional experience Create up to 3 bullets maxIntuitive individual with an ability to manage change and proven time management Proven interpersonal skills while contributing to team effort by accomplishing related results as needed Up-to-date technical knowledge by attending educational workshops, reviewing publications (encouraging then to focus on required skills)

Work Mode: Hybrid (3-5 Days from Office) Job Summary: Looking for a Senior Data Engineer to work on building and managing scalable data pipelines in a GCP environment. Responsibilities: Design, develop, and maintain scalable data pipelines. Manage ETL/ELT workflows from various data sources. Ensure data governance, quality, and transformation. Collaborate with stakeholders to meet data needs. Optimize for cost, scalability, and performance. Required Skill Set: Languages & Frameworks: Python, PySpark, SQL Orchestration Tools: Airflow / GCP Composer Streaming: Kafka Cloud Platform: Google Cloud Platform (Dataproc, BigQuery, Compute, Looker) Concepts: Data Modeling, Data Warehousing, CI/CD

Job Summary: Looking for a Senior Data Engineer to work on building and managing scalable data pipelines in a GCP environment. Responsibilities: Design, develop, and maintain scalable data pipelines. Manage ETL/ELT workflows from various data sources. Ensure data governance, quality, and transformation. Collaborate with stakeholders to meet data needs. Optimize for cost, scalability, and performance. Required Skill Set: Languages & Frameworks: Python, PySpark, SQL Orchestration Tools: Airflow / GCP Composer Streaming: Kafka Cloud Platform: Google Cloud Platform (Dataproc, BigQuery, Compute, Looker) Concepts: Data Modeling, Data Warehousing, CI/CD

Job Title: Software Engineer III - Core Engineer III Location: Chennai Work Type: Hybrid Overview: TekWissen is a global workforce management provider throughout India and many other countries in the world. The below clientis a global company with shared ideals and a deep sense of family. From our earliest days as a pioneer of modern transportation, we have sought to make the world a better place one that benefits lives, communities and the planet Position Description: MUST HAVE: Performs DevSecOps and SRE adoption for existing/new SpringBoot/Angular/REACT based applications to deploy in CaaS/GCP CloudRun. 5+ years of experience in Java/SpringBoot/Angular based development projects Hands-on experience in implementing Security tools like Cycode (SAST),42 crunch(Open API standard), Sonarqube, FOSSA(SCA), Kyverno(Policy as code) for existing/new applications Experience on programming languages such as Python or R, with experience in AI/ML libraries 2+ years of experience working in Agile projects Experience on Development, Operational, Monitoring tools with knowledge on SRE will be advantageous Research new tools and technologies, influences and implements enterprise technology shifts and new trends impacting Client Dev Ops Strategy Consults to resolve complex application DevSecOps challenges/issues. Consults with teams on applying relevant DevSecOps Integration tools suitable for the application. Experience in handling Source control Tool like Git Strong Experience in developing and troubleshooting Build scripts like GRADLE Experience in Developing and maintaining pipeline tools like Cloud build/Tekton Strong Experience in handling and troubleshooting Dev Sec Ops tools like Cycode, FOSSA, 42 crunch, SonarQube Strong Experience in integrating Dev Ops Testing tools like Selenium, Playwright NICE TO HAVE Experience in handling Monitoring & Logging tools like Dynatrace, Splunk and Grafana • Experience in deploying applications in OpenShift CaaS, GCP Cloud Run Experience and understanding of various LLMs and GenAI capabilities to bring ideas to production for enablers, automation etc. Experience with DevSecOps practices and integrating security into CI/CD pipelines. (including Jenkins/Tekton, GitLab CI, GitHub Actions), infrastructure-as-code (Terraform, Ansible, CloudFormation), and automation. Skills Required: Full Stack Java Developer, DevOps, Google Cloud Platform - Biq Query, Data Flow, Dataproc, Data Fusion, TERRAFORM, Tekton,Cloud SQL, AIRFLOW, POSTGRES, Airflow PySpark, Python, API Skills Preferred: Dynatrace, AI/ML, AIPGEE Experience Required: Engineer 3 Exp: Prac. In 2 coding lang. or adv. Prac. in 1 lang. 6+ years in IT; 4+ years in development Education Required: Bachelor's Degree TekWissen Group is an equal opportunity employer supporting workforce diversity.

Work Mode: Hybrid (35 Days from Office) Experience: 5+ Years Job Summary: Looking for a Senior Data Engineer to work on building and managing scalable data pipelines in a GCP environment. Responsibilities: Design, develop, and maintain scalable data pipelines. Manage ETL/ELT workflows from various data sources. Ensure data governance, quality, and transformation. Collaborate with stakeholders to meet data needs. Optimize for cost, scalability, and performance. Required Skill Set: Languages & Frameworks: Python, PySpark, SQL Orchestration Tools: Airflow / GCP Composer Streaming: Kafka Cloud Platform: Google Cloud Platform (Dataproc, BigQuery, Compute, Looker) Concepts: Data Modeling, Data Warehousing, CI/CD

Roles and Responsibilities Experience - 10+ Years Design, develop, and maintain large-scale data pipelines using BigQuery, Dataproc, and GCP. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs. Develop complex SQL queries to extract insights from massive datasets stored in BigQuery.

Role & responsibilities 8+ years experience Should have experience in GCP BigQuery, DataProc(PySpark) Should have experience on Informatica Preferred candidate profile Job Description :Tech Lead Data Engineer Experience : 8+ Years Work location: Chennai, Bangalore, Hyderabad, Pune Shift Timing : 2 to 11 PM Interview process : L1, L2 & Client round

Were Hiring: Senior Data Engineer GCP | Databricks | E-commerce Domain Work Locations : Chennai | Bangalore | Hyderabad | Gurugram | Jaipur | Pune | Bhopal Experience : 7–8+ Years Shift Timing : 2 PM – 10 PM IST Work Mode : Hybrid – 3 days/week from office Only Immediate Joiners (0–15 Days Notice) Are you a seasoned Data Engineer passionate about building next-gen data platforms? We’re hiring for an offshore role supporting one of our top global e-commerce clients. You'll work on large-scale consumer data and cloud-native architectures, helping create impactful data products. Join Xebia and become part of a high-performance team working with the latest technologies in data engineering. Role Responsibilities : Design and build robust, scalable data pipelines Develop data products and solutions using GCP , BigQuery , Databricks , DBT , Airflow , and Spark Handle ETL/ELT processes across structured and unstructured data Work with both SQL and NoSQL databases (e.g., PostgreSQL, MongoDB, Cassandra) Model and manage data lakes , data warehouses, and schema designs Collaborate cross-functionally with data consumers and product teams Contribute to best practices in data engineering, including unit testing and documentation Required Skills : 7–8+ years of relevant experience in data engineering and architecture Mandatory hands-on expertise with Google Cloud Platform (GCP) Proficiency in Python , SQL , and cloud-based data engineering frameworks Experience with Databricks , Airflow , DBT , BigQuery , and Spark Strong knowledge of data modeling and pipeline orchestration Solid communication and stakeholder collaboration skills Good to Have : Experience working in eCommerce or consumer data domains Exposure to data visualization tools (Tableau, Looker) Knowledge of machine learning workflows Certifications in GCP or other cloud platforms How to Apply : Send your updated CV to vijay.s@xebia.com along with these details: Full Name Total Experience Current CTC Expected CTC Current Location Preferred Xebia Location Notice Period / Last Working Day Primary Skills LinkedIn Profile Join us and power digital transformation in the consumer-tech world. Note : This is a hybrid role – 3 days/week from office is mandatory. #Xebia #HiringNow #SeniorDataEngineer #GCPJobs #Databricks #BigQuery #Airflow #DBT #PythonJobs #HybridJobs #EcommerceData #ImmediateJoiners #IndiaJobs #CloudDataEngineering

Data engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets. - Grade Specific The role support the team in building and maintaining data infrastructure and systems within an organization. Skills (competencies) Ab Initio Agile (Software Development Framework) Apache Hadoop AWS Airflow AWS Athena AWS Code Pipeline AWS EFS AWS EMR AWS Redshift AWS S3 Azure ADLS Gen2 Azure Data Factory Azure Data Lake Storage Azure Databricks Azure Event Hub Azure Stream Analytics Azure Sunapse Bitbucket Change Management Client Centricity Collaboration Continuous Integration and Continuous Delivery (CI/CD) Data Architecture Patterns Data Format Analysis Data Governance Data Modeling Data Validation Data Vault Modeling Database Schema Design Decision-Making DevOps Dimensional Modeling GCP Big Table GCP BigQuery GCP Cloud Storage GCP DataFlow GCP DataProc Git Google Big Tabel Google Data Proc Greenplum HQL IBM Data Stage IBM DB2 Industry Standard Data Modeling (FSLDM) Industry Standard Data Modeling (IBM FSDM)) Influencing Informatica IICS Inmon methodology JavaScript Jenkins Kimball Linux - Redhat Negotiation Netezza NewSQL Oracle Exadata Performance Tuning Perl Platform Update Management Project Management PySpark Python R RDD Optimization SantOs SaS Scala Spark Shell Script Snowflake SPARK SPARK Code Optimization SQL Stakeholder Management Sun Solaris Synapse Talend Teradata Time Management Ubuntu Vendor Management

Capgemini Invent Capgemini Invent is the digital innovation, consulting and transformation brand of the Capgemini Group, a global business line that combines market leading expertise in strategy, technology, data science and creative design, to help CxOs envision and build whats next for their businesses. Your Role Should have developed/Worked for atleast 1 Gen AI project. Has data pipeline implementation experience with any of these cloud providers - AWS, Azure, GCP. Experience with cloud storage, cloud database, cloud data warehousing and Data lake solutions like Snowflake, Big query, AWS Redshift, ADLS, S3. Has good knowledge of cloud compute services and load balancing. Has good knowledge of cloud identity management, authentication and authorization. Proficiency in using cloud utility functions such as AWS lambda, AWS step functions, Cloud Run, Cloud functions, Azure functions. Experience in using cloud data integration services for structured, semi structured and unstructured data such as Azure Databricks, Azure Data Factory, Azure Synapse Analytics, AWS Glue, AWS EMR, Dataflow, Dataproc. Your Profile Good knowledge of Infra capacity sizing, costing of cloud services to drive optimized solution architecture, leading to optimal infra investment vs performance and scaling. Able to contribute to making architectural choices using various cloud services and solution methodologies. Expertise in programming using python. Very good knowledge of cloud Dev-ops practices such as infrastructure as code, CI/CD components, and automated deployments on cloud. Must understand networking, security, design principles and best practices in cloud. What you will love about working here We recognize the significance of flexible work arrangements to provide support. Be it remote work, or flexible work hours, you will get an environment to maintain healthy work life balance. At the heart of our mission is your career growth. Our array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities. Equip yourself with valuable certifications in the latest technologies such as Generative AI. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of 22.5 billion.

Data engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets. - Grade Specific The role support the team in building and maintaining data infrastructure and systems within an organization. Skills (competencies) Ab Initio Agile (Software Development Framework) Apache Hadoop AWS Airflow AWS Athena AWS Code Pipeline AWS EFS AWS EMR AWS Redshift AWS S3 Azure ADLS Gen2 Azure Data Factory Azure Data Lake Storage Azure Databricks Azure Event Hub Azure Stream Analytics Azure Sunapse Bitbucket Change Management Client Centricity Collaboration Continuous Integration and Continuous Delivery (CI/CD) Data Architecture Patterns Data Format Analysis Data Governance Data Modeling Data Validation Data Vault Modeling Database Schema Design Decision-Making DevOps Dimensional Modeling GCP Big Table GCP BigQuery GCP Cloud Storage GCP DataFlow GCP DataProc Git Google Big Tabel Google Data Proc Greenplum HQL IBM Data Stage IBM DB2 Industry Standard Data Modeling (FSLDM) Industry Standard Data Modeling (IBM FSDM)) Influencing Informatica IICS Inmon methodology JavaScript Jenkins Kimball Linux - Redhat Negotiation Netezza NewSQL Oracle Exadata Performance Tuning Perl Platform Update Management Project Management PySpark Python R RDD Optimization SantOs SaS Scala Spark Shell Script Snowflake SPARK SPARK Code Optimization SQL Stakeholder Management Sun Solaris Synapse Talend Teradata Time Management Ubuntu Vendor Management

Capgemini Invent Capgemini Invent is the digital innovation, consulting and transformation brand of the Capgemini Group, a global business line that combines market leading expertise in strategy, technology, data science and creative design, to help CxOs envision and build whats next for their businesses. Your Role Should have developed/Worked for atleast 1 Gen AI project. Has data pipeline implementation experience with any of these cloud providers - AWS, Azure, GCP. Experience with cloud storage, cloud database, cloud data warehousing and Data lake solutions like Snowflake, Big query, AWS Redshift, ADLS, S3. Has good knowledge of cloud compute services and load balancing. Has good knowledge of cloud identity management, authentication and authorization. Proficiency in using cloud utility functions such as AWS lambda, AWS step functions, Cloud Run, Cloud functions, Azure functions. Experience in using cloud data integration services for structured, semi structured and unstructured data such as Azure Databricks, Azure Data Factory, Azure Synapse Analytics, AWS Glue, AWS EMR, Dataflow, Dataproc. Your Profile Good knowledge of Infra capacity sizing, costing of cloud services to drive optimized solution architecture, leading to optimal infra investment vs performance and scaling. Able to contribute to making architectural choices using various cloud services and solution methodologies. Expertise in programming using python. Very good knowledge of cloud Dev-ops practices such as infrastructure as code, CI/CD components, and automated deployments on cloud. Must understand networking, security, design principles and best practices in cloud. What you will love about working here We recognize the significance of flexible work arrangements to provide support. Be it remote work, or flexible work hours, you will get an environment to maintain healthy work life balance. At the heart of our mission is your career growth. Our array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities. Equip yourself with valuable certifications in the latest technologies such as Generative AI. About Capgemini Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of 22.5 billion.

A GCP Data Engineering Architect designs, implements, and manages scalable data solutions on Google Cloud Platform (GCP). They are responsible for defining data architecture, ensuring data quality and security, and optimizing data pipelines for various use cases, including data warehousing, big data processing, and real-time analytics. This role involves collaborating with stakeholders, mentoring junior engineers, and staying up-to-date with the latest GCP technologies. Skills Required: 12~16 years' experience in IT or professional services experience in IT delivery or large-scale IT data engineering projects. Data Pipeline Development: Building and optimizing data pipelines using GCP services like BigQuery, Dataflow, Pub/Sub, Dataproc, Cloud Storage, Cloud Composer, and Cloud Run. Designing and Architecting Data Solutions: Developing end-to-end data solutions on GCP, including data models, storage strategies, data ingestion, processing, and consumption frameworks. Data Security and Governance: Implementing data security frameworks, establishing data governance policies, and ensuring compliance with data quality and privacy standards. Data Warehousing and Big Data: Solid understanding of data warehousing concepts, big data processing frameworks, and ETL/ELT processes Mentoring and Leadership: Providing technical guidance, mentoring junior team members, and contributing to the overall data engineering strategy. Staying Updated: Keeping abreast of the latest GCP services, data architecture trends, and best practices. Expert knowledge in SQL development. Required Skills Key Skills - Data engineering Architecture, GCP services like BigQuery, Dataflow, Pub/Sub, Dataproc, Cloud Storage, Cloud Composer, and Cloud Run, technical mentoring