15216 Airflow Jobs

Project description Support one of the top Australian banks as they seek to modernise their data and analytics platform. You will be working directly with IT and business stakeholders in Data and Platform team to implement banks data strategy to become the best AI bank of the world. Responsibilities We are looking for a Software Engineer with strong experience in designing, building, and maintaining production-grade software systems. The role focuses on automation, quality, and resilience across the software development lifecycle, with end-to-end ownership of features from design through deployment and monitoring. Key Responsibilities Design and implement robust, scalable software solutions....

Project description We are looking for a skilled Data Engineer with strong experience in AWS and Apache Airflow to join a dynamic data team. You will be responsible for building and maintaining scalable data pipelines, orchestrating workflows, and ensuring data quality and availability across platforms. Responsibilities ?Data Pipeline DevelopmentDesign, build, and maintain robust, scalable, and efficient ETL/ELT pipelines for ingesting, transforming, and loading data from various sources into our data lake and data warehouse. ?AWS ExpertiseDevelop and manage data solutions using a wide range of AWS services, including but not limited to ?StorageS3 (for data lakes), RDS (PostgreSQL, MySQL), R...

Project description We have an ambitious goal to migrate a legacy system written in HLASM (High-Level Assembler) from the mainframe to a cloud-based Java environment for one of the largest banks in the USA. Responsibilities We are looking for an experienced Senior DevOps who can Design, implement, and maintain scalable cloud infrastructure on AWS, leveraging services like ECS, AWS Managed workflows for Apache Airflow, Lambda, and S3; Develop and manage infrastructure-as-code using Terraform for efficient environment management; Build and maintain CI/CD pipelines using tools such as Jenkins, Jules or Spinnaker; Manage ECS clusters, including configuration, deployment, and monitoring (Airflow ...

Job Title: GCP Data Engineer Location: Bangalore, Chennai, Pune, Hyderabad and Vadodara Experience: 7 Years - 10 Years Notice Period: Immediate Joiners ( Preferred) Key Responsibilities: Design, develop, and maintain data ingestion and transformation pipelines on GCP using BigQuery, PySpark, and Airflow DAGs. Implement data processing and analytics workflows aligned with performance, scalability, and reliability standards. Collaborate with cross-functional teams (data architects, analysts, and application developers) to deliver integrated solutions. Optimize BigQuery SQL scripts for efficiency and cost management. Develop reusable data processing components and automation frameworks. (Option...

Work Location - ~ - PAN INDIA-Contract to Hire Skill Required - Digital : Amazon Web Service(AWS) Cloud Computing~Digital : Scala Experience Range 4 to 6 Job Description – Role Expectation: We need the Scala Developer not Big Data OR Data Engineers, please eliminate the candidate who are working on Apache Spark, Airflow, Kafka and Airflow. Please look for the candidates who are into Scala development, with Microservices, Akka, REST Api experience. Roles and responsibilities – • 3 + years of experience in Scala backend development.

Job Description: Develop, optimize, and maintain end-to-end ETL/ELT pipelines using Databricks , PySpark , and Delta Lake for high-performance data processing. Design and implement scalable data lakehouse architectures , including bronzesilver–gold layers following the Medallion Architecture . Perform advanced data transformations , cleansing, and enrichment to ensure data quality, consistency, and business readiness. Work closely with stakeholders to gather requirements and translate them into efficient Databricks workflows and data models. Implement and manage job orchestration , monitoring, and debugging using Databricks Jobs, Workflows, or Airflow. Optimize PySpark code, cluster configur...

Cloud Data Architect Role: T0 The Ideal Professional For This Cloud Architect Role Will Have a passion for design, technology, analysis, collaboration, agility, and planning, along with a drive for continuous improvement and innovation. Exhibit expertise in managing high-volume data projects that leverage Cloud Platforms, Data Warehouse reporting and BI Tools, and the development of relational databases. Research, identify, and internally market enabling data management technologies based on business and end-user requirements. Seek ways to apply new technology to business processes with a focus on modernizing the approach to data management. Consult with technical subject matter experts and ...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Google BigQuery Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project ...

About the Role We are looking for a detail-oriented Data Analyst to collect, analyze, and interpret large datasets to help drive informed business decisions. You will work closely with cross-functional teams to provide insights, dashboards, and data-driven recommendations. Key Responsibilities Collect, clean, and validate data from multiple sources. Analyze trends, patterns, and insights to support business goals. Build dashboards and reports using tools like Power BI, Tableau, or Looker. Create data visualizations that communicate complex insights clearly. Work with stakeholders to understand data needs and deliver actionable insights. Develop SQL queries and scripts for data extraction and...

Title: Snowflake + dbt Data Architect Location: Kochi Experience: 7+ years in data engineering/architecture; 3+ years hands-on with Snowflake & dbt Role summary You’ll own the technical design and delivery of a HIPAA-sensitive Snowflake + dbt data platform for a healthcare enterprise. You’ll translate business needs into a secure, cost-efficient, maintainable data architecture and lead two developers to implement it. Key responsibilities Lead end-to-end design of the Snowflake data platform: schema design, warehouses, resource monitors, roles, and sharing strategies. Define and enforce dbt architecture & conventions (models, macros, seeds, tests, CI/CD). Design secure ingestion patterns (Sno...

Greetings from HCLTECH! We are hiring for GCP Data Engineer role for the location #Chennai #Hyderabad Role: GCP Data Engineer Experience: 4 Years to 13 Years Location: Chennai, Hyderabad Interview Mode: F2F Date: 22- Nov-2025 Job Description: Mandatory Skill: GCP BigQuery Cloud Composer or Airflow (Anyone is fine) Dataflow or Dataproc or Datafusion (Anyone is fine) Python or Pyspark (Anyone is fine) DBT SQL Nice to have: GKE, Spanner, Harness, GEN AI, looker Interested candidates, please share the details using the link GCP DE – Fill out form Note: Only candidates with GCP experience are eligible to apply

You are passionate about quality and how customers experience the products you test. You have the ability to create, maintain and execute test plans in order to verify requirements. As a Quality Engineer at Equifax, you will be a catalyst in both the development and the testing of high priority initiatives. You will develop and test new products to support technology operations while maintaining exemplary standards. As a collaborative member of the team, you will deliver QA services (code quality, testing services, performance engineering, development collaboration and continuous integration). You will conduct quality control tests in order to ensure full compliance with specified standards ...

Title: Snowflake + dbt Data Architect Location: Kochi Experience: 7+ years in data engineering/architecture; 3+ years hands-on with Snowflake & dbt Role summary You’ll own the technical design and delivery of a HIPAA-sensitive Snowflake + dbt data platform for a healthcare enterprise. You’ll translate business needs into a secure, cost-efficient, maintainable data architecture and lead two developers to implement it. Key responsibilities Lead end-to-end design of the Snowflake data platform: schema design, warehouses, resource monitors, roles, and sharing strategies. Define and enforce dbt architecture & conventions (models, macros, seeds, tests, CI/CD). Design secure ingestion patterns (Sno...

Description DAZN Hyderabad is seeking a talented Senior Data Engineer with strong experience in modern data platforms and ETL/ELT development to support our global data initiatives. In this role, you will design and optimize data pipelines, work with Snowflake, DBT, and Airflow, and collaborate with cross-functional teams to deliver high-quality, reliable data solutions that power insights across the organization. Key Responsibilities Design, build, and maintain scalable ETL/ELT pipelines to support analytics, reporting, and data-driven product features. Develop and optimize complex SQL queries and contribute to performance tuning and data quality improvements. Work extensively with Snowflak...

At Lilly, we unite caring with discovery to make life better for people around the world. We are a global healthcare leader headquartered in Indianapolis, Indiana. Our employees around the world work to discover and bring life-changing medicines to those who need them, improve the understanding and management of disease, and give back to our communities through philanthropy and volunteerism. We give our best effort to our work, and we put people first. We’re looking for people who are determined to make life better for people around the world. About the Tech@Lilly Organization: Tech@Lilly builds and maintains capabilities using cutting edge technologies like most prominent tech companies. Wh...

About the Role We are seeking an experienced Data Engineer Lead to design, develop, and maintain scalable data solutions on Azure and Databricks as part of our enterprise data modernization initiatives. The ideal candidate will have a strong background in data pipeline development, data integration frameworks , and cloud-based data engineering , with deep expertise in tools such as Databricks, Azure Data Factory, Alteryx, Ab Initio, Talend, and Informatica . This role will lead the design and delivery of high-performance, governed data architectures for large-scale enterprise clients, driving data reliability, compliance, and analytics readiness. Key Responsibilities Design and implement sca...

Company Description: Quanteon Solutions, established in 2015, transforms complex challenges into scalable, smart solutions for industries like Healthcare, FinTech, E-Commerce, Insurance, and Sports. Specializing in Custom Web & Mobile Application Development, Salesforce Implementation, Cloud Hosting & Migration, RPA, Data Analytics, Machine Learning, and Generative AI, we deliver secure and high-performing solutions tailored for modern businesses. With over 300 successful projects, we enable startups, enterprises, and brands to optimize workflows, scale operations, and enhance customer engagement. Our commitment to innovation, collaboration, and security ensures businesses stay ahead in a dy...

Position Overview We are looking for an AI Data Engineer with 2+ years of hands-on experience in data engineering, vector databases, LLM/AI integrations, and Python-based automation. The ideal candidate should have strong knowledge of Open AI , AWS Bedrock , embeddings , Flowise , prompt engineering , and building scalable RAG (Retrieval-Augmented Generation) pipelines. Key Responsibilities : AI & LLM Integrations Integrate and optimize OpenAI, GPT models, AWS Bedrock models (Claude, Titan, etc.) for production workflows. Build and maintain Retrieval-Augmented Generation (RAG) systems using vector search. Develop prompt engineering strategies , structured prompts, and dynamic contextual prom...

Expedia Group brands power global travel for everyone, everywhere. We design cutting-edge tech to make travel smoother and more memorable, and we create groundbreaking solutions for our partners. Our diverse, vibrant, and welcoming community is essential in driving our success. Why Join Us? To shape the future of travel, people must come first. Guided by our Values and Leadership Agreements, we foster an open culture where everyone belongs, differences are celebrated and know that when one of us wins, we all win. We provide a full benefits package, including exciting travel perks, generous time-off, parental leave, a flexible work model (with some pretty cool offices), and career development...

Lead Data Engineer Location Gurugram - Haryana, India FORVIA HELLA is a listed international automotive supplier. As a company of the FORVIA Group, FORVIA HELLA stands for high-performance lighting technology and vehicle electronics and, with the Lifecycle Solutions Business Group, also covers a broad service and product portfolio for the spare parts and workshop business as well as for manufacturers of special vehicles. With currently around 36,500 employees at over 125 locations, the Company is active worldwide and generated adjusted sales of €8.1 billion in fiscal year 2024. YOUR TASKS We’re hiring a Lead Data Engineer to join our Data Engineering team as a high-impact individual contribu...

Ninja Van is a late-stage logtech startup that is disrupting a massive industry with innovation and cutting edge technology. Launched 2014 in Singapore, we have grown rapidly to become one of Southeast Asia's largest and fastest-growing express logistics companies. Since our inception, we’ve delivered to 100 million different customers across the region with added predictability, flexibility and convenience. Join us in our mission to connect shippers and shoppers across Southeast Asia to a world of new possibilities. More About Us We process 250 million API requests and 3TB of data every day. We deliver more than 2 million parcels every day. 100% network coverage with 2600+ hubs and stations...

Ninja Van is a late-stage logtech startup that is disrupting a massive industry with innovation and cutting edge technology. Launched 2014 in Singapore, we have grown rapidly to become one of Southeast Asia's largest and fastest-growing express logistics companies. Since our inception, we’ve delivered to 100 million different customers across the region with added predictability, flexibility and convenience. Join us in our mission to connect shippers and shoppers across Southeast Asia to a world of new possibilities. More About Us We process 250 million API requests and 3TB of data every day. We deliver more than 2 million parcels every day. 100% network coverage with 2600+ hubs and stations...

Job Title: Data Engineering Solutions Architect Location: Hyderabad Job Type: Full-time Job Description We are looking for an experienced Data Engineering Solutions Architect to join our growing Data Practice. The ideal candidate will have 8–12 years of hands-on experience designing, architecting, and delivering large-scale data warehousing, data lake, ETL, and reporting solutions across modern and traditional data platforms. You will play a key role in defining scalable, secure, and cost-effective architectures that enable advanced analytics and AI-driven insights for our clients. This role demands a balance of technical depth, solution leadership, and consulting mindset - helping customers...

Job Description: Experience : 4 to 6 years of experience in Data Engineering or related roles. Required Knowledge 1. Snowflake (Data warehousing) 2. DBT (Data transformation) 3. Airflow (Workflow orchestration) Responsibilities  Develop and maintain data pipelines using Snowflake, DBT, and Airflow.  Design and implement data integration strategies.  Optimize ETL processes and ensure clean, reliable data.  Collaborate with cross-functional teams to meet data requirements.  Monitor and troubleshoot data workflows to ensure performance. Job Type: Full-time Pay: ₹813,891.70 - ₹1,343,523.14 per year Work Location: In person

Company Description Zyphex Tech is a leading IT services and consulting company specializing in Artificial Intelligence, web development, and digital transformation. With a focus on advanced AI applications and custom development, we deliver scalable, secure, and innovative solutions tailored to meet diverse client needs. Our expertise spans machine learning, data analytics, DevOps engineering, and solution architecture to help businesses achieve operational excellence. We partner with startups, SMEs, and enterprises across industries to drive growth, streamline processes, and transform customer experiences with cutting-edge technology. Job Title: AI Tech Lead Location: India, Remote Experie...