224 Cloud Composer Jobs - Page 7

Experience Level: 46 years Note - Video of self Introduction of 3mins. Key Responsibilities Design, build, and maintain scalable and reliable data pipelines using GCP services such as: BigQuery, Cloud Dataflow, Pub/Sub, Cloud Storage, Cloud Composer (Airflow). Implement data ingestion from diverse sources (e.g., APIs, databases, files) into cloud data lake/warehouse. Optimize data processing workflows for performance, reliability, and cost efficiency. Develop and manage ETL/ELT processes using tools like Dataform, dbt, or custom scripts. Collaborate with data scientists, analysts, and stakeholders to deliver clean, well-documented datasets. Implement data quality, governance, and observabili...

Role & responsibilities Job Summary: As a GCP Data Engineer you will be responsible for developing, optimizing, and maintaining data pipelines and infrastructure. Your expertise in SQL and Python will be instrumental in managing and transforming data, while your familiarity with cloud technologies will be considered an asset as we explore opportunities to enhance data engineering processes. Job Description: Building scalable Data Pipelines Design, implement, and maintain end-to-end data pipelines to efficiently extract, transform, and load (ETL) data from diverse sources. Ensure data pipelines are reliable, scalable, and performance oriented. SQL Expertise: Write and optimize complex SQL que...

Role Overview: You are required to work as a GCP Data Architect with a total experience of 12+ years. Your relevant experience for engagement should be 10 years. Your primary responsibilities will include maintaining architecture principles, guidelines, and standards, data warehousing, programming in Python/Java, working with Big Data, Data Analytics, and GCP Services. You will be responsible for designing and implementing solutions in various technology domains related to Google Cloud Platform Data Components like BigQuery, BigTable, CloudSQL, Dataproc, Data Flow, Data Fusion, etc. Key Responsibilities: - Maintain architecture principles, guidelines, and standards - Work on Data Warehousing...

Join Us At Vodafone, we're not just shaping the future of connectivity for our customers - we're shaping the future for everyone who joins our team. When you work with us, you're part of a global mission to connect people, solve complex challenges, and create a sustainable and more inclusive world. If you want to grow your career whilst finding the perfect balance between work and life, Vodafone offers the opportunities to help you belong and make a real impact. What you'll do Conduct end-to-end impact assessments across all subject areas for new demands. Create and maintain comprehensive data architecture documentation including data models, flow diagrams, and technical specifications. Desi...

As a GCP Data Engineer, you will play a crucial role in the development, optimization, and maintenance of data pipelines and infrastructure. Your proficiency in SQL and Python will be pivotal in the management and transformation of data. Moreover, your familiarity with cloud technologies will be highly beneficial as we strive to improve our data engineering processes. You will be responsible for building scalable data pipelines. This involves designing, implementing, and maintaining end-to-end data pipelines to efficiently extract, transform, and load (ETL) data from various sources. It is essential to ensure that these data pipelines are reliable, scalable, and performance-oriented. Your ex...

We are looking for an experienced GCP Data Engineer with 510 years of experience in Google Cloud Platform (GCP) services and Big Data Analytics solutions . This is an exciting opportunity for professionals passionate about designing and implementing scalable data engineering solutions while working on advanced cloud-based projects. Key Responsibilities:- Design, build, and optimize data pipelines using BigQuery, Dataflow, DataProc, and Cloud Composer . Implement Big Data Analytics solutions leveraging Hadoop, Hive, and Spark on GCP. Write efficient queries and scripts with SQL, PySpark, and Python for large-scale data processing. Work with orchestration and messaging tools like Airflow, Kafk...

Job Description; Technical Skills; Top skills for this positions is : Google Cloud Platform (Composer, Big Query, Airflow, DataProc, Data Flow, GCS) Data Warehousing knowledge Hands on experience in Python language and SQL database. Analytical technical skills to be able to predict the consequences of configuration changes (impact analysis), to identify root causes that are not obvious and to understand the business requirements. Excellent communication with different stakeholders (business, technical, project) Good understading of the overall Big Data and Data Science ecosystem Experience with buiding and deploying containers as services using Swarm/Kubernetes Good understanding of containe...

Position Title Lead Infrastructure Engineer- Integration Function/Group Digital and Technology Location Mumbai Shift Timing Regular Role Reports to D&T Manager - Integration Remote/Hybrid/in-Office Hybrid ABOUT GENERAL MILLS We make foodthe world loves: 100 brands. In 100 countries. Across six continents. With iconic brands like Cheerios, Pillsbury, Betty Crocker, Nature Valley, and Hagen-Dazs, we've been serving up food the world loves for 155 years (and counting). Each of our brands has a unique story to tell. How we make our food is as important as the food we make. Our values are baked into our legacy and continue to accelerate us into the future as an innovative force for good. General ...

Job description Some careers shine brighter than others. If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to...

This role is for a GCP Data Engineer who can build cloud analytics platforms to meet expanding business requirements with speed and quality using lean Agile practices. You will work on analysing and manipulating large datasets supporting the enterprise by activating data assets to support Enabling Platforms and Analytics in the GCP. You will be responsible for designing the transformation and modernization on GCP. Experience with large scale solutions and operationalizing of data warehouses, data lakes and analytics platforms on Google Cloud Platform or other cloud environment is a must. We are looking for candidates who have a broad set of technology skills across these areas and who can de...

As a qualified candidate for this role, you should have in-depth expertise in Google Cloud Platform (GCP) services such as Pubsub, BigQuery, Airflow, Data Proc, Cloud Composer, and Google Cloud Storage (GCS). Additionally, proficiency in DataFlow and Java is a must for this position. Experience with Kafka would be considered a plus. Your responsibilities will include working with these technologies to design, develop, and maintain scalable and efficient data processing systems. If you meet these requirements and are eager to work in a dynamic and innovative environment, we look forward to reviewing your application.,

Position Title Sr Infrastructure Engineer- Integration Function/Group Digital and Technology Location Mumbai Shift Timing Regular Role Reports to D&T Manager - Integration Remote/Hybrid/in-Office Hybrid ABOUT GENERAL MILLS We make foodthe world loves: 100 brands. In 100 countries. Across six continents. With iconic brands like Cheerios, Pillsbury, Betty Crocker, Nature Valley, and Hagen-Dazs, we've been serving up food the world loves for 155 years (and counting). Each of our brands has a unique story to tell. How we make our food is as important as the food we make. Our values are baked into our legacy and continue to accelerate us into the future as an innovative force for good. General Mi...

Position Description: Representing the Ford Credit (FC) Data Engineering Organization as a Google Cloud Platform (GCP) Data Engineer, specializing in migration and transformation, you will be a developer part of a global team to build a complex Datawarehouse in the Google Cloud Platform. This role involves designing, implementing, and optimizing data pipelines, ensuring data integrity during migration, and leveraging GCP services to enhance data transformation processes for scalability and efficiency. This role is for a GCP Data Engineer who can build cloud analytics platforms to meet expanding business requirements with speed and quality using lean Agile practices. You will work on analyzin...

You have over 10 years of experience in data engineering, specializing in cloud-based solutions. Your role involves designing solutions, reviewing team work, and providing guidance. Proficiency in Google Cloud Platform (GCP) and its various data services such as BigQuery, DBT & Streaming, Dataflow, Pub/Sub, Cloud Storage, and Cloud Composer is essential. Your track record should demonstrate your ability to create scalable data pipelines and architectures. Experience with ETL tools, processes, and implementing ETL processes to transfer data to GCP warehouses like BigQuery is required. Your technical skills should include proficiency in DBT & Streaming, Dataflow, Cloud Storage, Cloud Composer,...

As a Google Cloud DevOps Engineer specializing in Terraform and CI/CD Pipeline, you will play a crucial role in provisioning GCP resources based on architectural designs that align with business objectives. Your responsibilities will include monitoring resource availability and usage metrics to provide guidelines for cost and performance optimization. You will be expected to assist IT and business users in resolving GCP service-related issues and provide guidance on cluster automation and migration approaches. Additionally, your role will involve provisioning GCP resources for data engineering and data science projects, including automated data ingestion, migration, and transformation. Key R...

As a Data Engineer at Aptiv, you will play a crucial role in designing, developing, and implementing a cost-effective, scalable, reusable, and secured Ingestion framework. Your primary responsibility will be to work closely with business leaders, stakeholders, and source system Subject Matter Experts (SMEs) to understand and define the business needs, translate them into technical specifications, and ingest data into Google Cloud Platform, specifically BigQuery. You will be involved in designing and implementing processes for data ingestion, transformation, storage, analysis, modeling, reporting, monitoring, availability, governance, and security of high volumes of structured and unstructure...

As a GCP Data Engineer specialized in Data Migration & Transformation, you will be responsible for designing and constructing robust, scalable data pipelines and architectures on Google Cloud Platform (GCP), particularly focusing on BigQuery. Your primary tasks will involve migrating and transforming large-scale data systems and datasets to GCP while emphasizing performance, scalability, and reliability. It will be crucial for you to automate data lineage extraction and ensure data integrity across various systems and platforms. Collaborating closely with architects and stakeholders, you will play a key role in implementing GCP-native and 3rd-party tools for data ingestion, integration, and ...

Role: GCP Data Engineer Experience: 5-9 Years Notice: 15 Days less Interview Mode: First Round Virtual/ Second Round Face to face (Mandate) Location: Bangalore Job Description Data Ingestion, Storage, Processing and Migration Acquire, cleanse, and ingest structured and unstructured data on the cloud platforms (in batch or real time) from internal and external data sources Combine data from disparate sources in to a single, unified, authoritative view of data (e.g., Data Lake) Create, maintain and provide test data to support fully automated testing Enable and support data movement from one system / service to another system / service. Reporting : Design, Develop and maintain high performance...

Data Engineer (ETL, Python, SQL, GCP) Position Overview Job Title: Data Engineer (ETL, Python, SQL, GCP) Corporate Title: AVP Location: Pune, India Role Description Engineer is responsible for developing and delivering elements of engineering solutions to accomplish business goals. Awareness is expected of the important engineering principles of the bank. Root cause analysis skills develop through addressing enhancements and fixes 2 products build reliability and resiliency into solutions through early testing peer reviews and automating the delivery life cycle. Successful candidate should be able to work independently on medium to large sized projects with strict deadlines. Successful candi...

You are urgently required to join as a Senior BigQuery Developer (Google Cloud Platform) with a minimum experience of 5-8 years in Hyderabad. In this role, you will be responsible for designing, developing, and maintaining robust, scalable data pipelines and advanced analytics solutions using BigQuery and other GCP-native services. Your primary focus will be on designing, developing, and optimizing BigQuery data warehouses and data marts to support analytical and business intelligence workloads. You will also need to implement data modeling and best practices for partitioning, clustering, and table design in BigQuery. Integration of BigQuery with tools such as Dataform, Airflow, Cloud Compos...

As a Data Engineer, you will be responsible for designing and developing scalable data engineering solutions using Google Cloud Platform (GCP) and PySpark. Your main focus will be on optimizing Spark jobs for performance, scalability, and efficient resource utilization. You will also be involved in developing, maintaining, and enhancing ETL pipelines using BigQuery, Apache Airflow, and Cloud Composer. Collaborating with data scientists, analysts, and DevOps teams to translate business requirements into technical solutions will be a key aspect of your role. Ensuring data integrity and security by implementing data governance, compliance, and security best practices will be crucial. Monitoring...

You will be responsible for designing and implementing cloud-native and hybrid solutions using GCP services such as Compute Engine, Kubernetes (GKE), Cloud Functions, BigQuery, Pub/Sub, Cloud SQL, and Cloud Storage. Additionally, you will define cloud adoption strategies, migration plans, and best practices for performance, security, and scalability. You will also be required to implement and manage Terraform, Cloud Deployment Manager, or Ansible for automated infrastructure provisioning. The ideal candidate should have expertise as a GCP data architect with network domain skills in GCP (DataProc, cloud composer, data flow, BQ), python, spark Py spark, and hands-on experience in the network ...

It is exciting to be part of a company where individuals genuinely believe in the purpose of their work. The commitment to infuse passion and customer-centricity into the business is unwavering. Fractal stands out as a key player in the field of Artificial Intelligence. The core mission of Fractal is to drive every human decision within the enterprise by integrating AI, engineering, and design to support the most esteemed Fortune 500 companies globally. Recognized as one of India's top workplaces by The Great Place to Work Institute, Fractal is at the forefront of innovation in Cloud, Data, and AI technologies, fueling digital transformation across enterprises at an unprecedented pace exceed...

As a Data Engineer at our company, you will be responsible for designing, developing, and maintaining scalable and efficient data pipelines using GCP services such as Dataflow, Cloud Composer (Airflow), and Pub/Sub. Your role will involve designing and implementing robust data models optimized for analytical and operational workloads within GCP data warehousing solutions like BigQuery. You will also be tasked with developing and implementing ETL processes to ingest, cleanse, transform, and load data from various sources into our data warehouse and other data stores on GCP. Furthermore, you will play a key role in building and managing data warehousing solutions on GCP, ensuring data integrit...

As a Consultant Delivery (Data Engineer) at Worldline, you will be an integral part of the Data Management team, contributing to a significant Move to Cloud (M2C) project. Your primary focus will be migrating our data infrastructure to the cloud and enhancing our data pipelines for improved performance and scalability. You will have the opportunity to work on a critical initiative that plays a key role in the organization's digital transformation. To excel in this role, you should hold a Bachelor's or Master's degree in Computer Science, Engineering, or a related field, along with a minimum of 5 years of experience as a Data Engineer. Your expertise should include a strong emphasis on cloud-...