974 Apache Airflow Jobs - Page 9

Key Responsibilities : - Dataiku Leadership : Drive data engineering initiatives with a strong emphasis on leveraging Dataiku capabilities for data preparation, analysis, visualization, and the deployment of data solutions. - Data Pipeline Development : Design, develop, and optimize robust and scalable data pipelines to support various business intelligence and advanced analytics projects. This includes developing and maintaining ETL/ELT processes to automate data extraction, transformation, and loading from diverse sources. - Data Modeling & Architecture : Apply expertise in data modeling techniques to design efficient and scalable database structures, ensuring data integrity and optimal pe...

Description : We are a technology consulting firm operating in Cloud Data Engineering and Analytics, helping enterprise customers build reliable, scalable data platforms and analytics products. Our teams deliver end-to-end data lakes, real-time streaming pipelines, and production-grade ML feature stores using Databricks and modern cloud data tooling. Role & Responsibilities : - Design, build, and maintain scalable batch and streaming ETL pipelines on Databricks using Delta Lake and Delta Live Tables (DLT). - Develop and optimize Spark/PySpark jobs for performance, cost-efficiency, and reliability; tune cluster sizing and autoscaling policies. - Implement data quality, observability, lineage ...

Python Developer (Lead Level) Location: Chennai Hybrid (3 Days a Week in Office) Experience: 8+ Years Employment Type: Contract (1 Year, Extendable) Notice Period: Immediate Joiners Only Role Overview: We are hiring a Senior Python Developer / Lead with strong experience in Python, SQL, AWS, and Airflow. The candidate will design, develop, and lead backend and data solutions in a cloud-based environment. Key Responsibilities: Lead the design and development of Python-based data pipelines and APIs. Work with AWS services (Lambda, S3, Glue, Redshift, EC2) for deployment and integration. Implement Airflow DAGs for data orchestration and automation. Optimize SQL queries and database performance....

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

We are a technology consulting firm operating in Cloud Data Engineering and Analytics, helping enterprise customers build reliable, scalable data platforms and analytics products. Our teams deliver end-to-end data lakes, real-time streaming pipelines, and production-grade ML feature stores using Databricks and modern cloud data tooling. Role & Responsibilities : - Design, build, and maintain scalable batch and streaming ETL pipelines on Databricks using Delta Lake and Delta Live Tables (DLT). - Develop and optimize Spark/PySpark jobs for performance, cost-efficiency, and reliability; tune cluster sizing and autoscaling policies. - Implement data quality, observability, lineage and monitoring...

We are seeking a Cloud Engineer to design and implement scalable, cloud-native workflows for GenAI, AI, and ML systems. This role focuses on building automated, event-driven pipelines using Apache Airflow, Azure Functions, and EventGrid, with full observability through Azure-native monitoring tools. The ideal candidate will have experience integrating these components into secure, production-grade environments using Infrastructure-as-Code (IaC) and CI/CD practices. Job Requirements Must-Have Skills & Experience Workflow Orchestration: Experience designing and managing end-to-end GenAI/AI/ML workflows using Apache Airflow or equivalent. Event-Driven Architecture: Proficiency in building event...

We are seeking an accomplished Senior DevOps Engineer with 1520 years of experience in cloud infrastructure management, automation, and CI/CD implementation The ideal candidate will architect, maintain, and optimize Azure cloud environments while ensuring scalability, performance, and reliability across enterprise applications Key responsibilities include managing Azure resources (VMs, AKS, API Management, Service Bus, Event Hub), administering Azure Kubernetes Service (AKS) and Apache Airflow, automating deployments via Python/Bash/PowerShell, and maintaining middleware systems like TIBCO The role demands strong expertise in CI/CD pipelines, Infrastructure as Code (Pulumi/ARM), and monitori...

As a Senior Data Engineer specializing in Snowflake and DBT, your role will involve designing, developing, and optimizing data pipelines using Airflow, Python, and DBT. You will be responsible for building and maintaining data models and data marts in Snowflake, implementing best practices in data transformation, testing, and documentation using DBT, and ensuring data quality, governance, and lineage across data pipelines. Additionally, you will collaborate with data analysts, scientists, and business stakeholders to understand data needs, optimize query performance and data storage in Snowflake, monitor and troubleshoot data pipeline issues proactively, and automate workflows and schedule t...

As an experienced Data Engineer at FIS, you will have the opportunity to work on challenging financial services and technology issues. You will be part of a team that values openness, collaboration, entrepreneurship, passion, and fun. **Role Overview:** You will be responsible for designing, developing, testing, and maintaining architectures such as databases and large-scale processing systems. Your key responsibilities will include: - Identifying and resolving database performance issues, capacity, and scalability - Overseeing the design, maintenance, and use of an Extract, Transform, Load (ETL) pipeline - Developing testable, reusable code in SQL and Python - Providing technical leadership...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

In this role at EY, you will be responsible for providing expert level data solutions using software to process, store, and serve data to others. You will test data quality, optimize data availability, and ensure that data pipelines are scalable, repeatable, and secure. Your role will involve utilizing deep analytical skills on internal and external data, as well as leading, instructing, and mentoring newer Data Engineering crew. Key Responsibilities: - Write ETL (Extract / Transform / Load) processes, design database systems, and develop tools for real-time and offline analytic processing. - Troubleshoot software and processes for data consistency and integrity, integrating complex and larg...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...

ABOUT THE ROLE We are seeking a seasoned Senior Data Engineer to drive the development and implementation of our data strategy. The ideal candidate possesses a strong blend of technical expertise and data-driven problem-solving skills. As a Senior Data Engineer, you will play a crucial role in designing, building, and optimizing our data pipelines and platforms while mentoring junior engineers. Roles & Responsibilities: Contribute to the design, development, and implementation of automations to monitor our data and cloud platforms for reliability, cost, and maintenance. Take ownership of platform management, managing scope, timelines, and risks. Ensure data quality and integrity through rigo...

Role Description: We are seeking an experienced Data Engineer for the development and implementation of our data strategy. The ideal candidate possesses a strong blend of technical expertise and data-driven problem-solving skills. As a Data Engineer, you will play a crucial role in building, and optimizing our data pipelines and platforms in a SAFE Agile product team. Roles & Responsibilities: Contribute to the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions. Deliver for data pipeline projects from development to deployment, managing, timelines, and risks. Ensure data quality and integrity through rigorous testing and monitoring. L...

As a Cloud Leader (Jr. Data Architect) with 7+ years of IT experience, you will be responsible for working on two Structural databases (SQL/Oracle/Postgres) and one NoSQL Database. You will collaborate with the Presales team to propose the best solution and architecture. Additionally, you should have design experience on BQ/Redshift/Synapse. Your role will involve managing the end-to-end product life cycle, from proposal to delivery. It is crucial to regularly check with the delivery team on architecture improvement. Key Responsibilities: - Work on two Structural databases (SQL/Oracle/Postgres) and one NoSQL Database - Collaborate with the Presales team to propose the best solution and archi...

As a Senior Data Engineer specializing in Snowflake DBT, your role will involve designing, developing, and optimizing data pipelines using Airflow, Python, and DBT. You will be responsible for building and maintaining data models and data marts in Snowflake, implementing best practices in data transformation, testing, and documentation using DBT, and collaborating with data analysts, scientists, and business stakeholders to understand data needs. It will be your duty to ensure data quality, governance, and lineage across data pipelines, optimize query performance and data storage in Snowflake, monitor and troubleshoot data pipeline issues proactively, and automate workflows and schedule task...

About The Role Job Title - Retail Specialized Data Scientist Level 9 SnC GN Data & AI Management Level:09 - Consultant Location:Bangalore / Gurgaon / Mumbai / Chennai / Pune / Hyderabad / Kolkata Must have skills: A solid understanding of retail industry dynamics, including key performance indicators (KPIs) such as sales trends, customer segmentation, inventory turnover, and promotions. Strong ability to communicate complex data insights to non-technical stakeholders, including senior management, marketing, and operational teams. Meticulous in ensuring data quality, accuracy, and consistency when handling large, complex datasets. Gather and clean data from various retail sources, such as sal...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Python (Programming Language), Apache Airflow Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :Seeking a forward-thinking professional with an AI-first mindset to design, develop, and deploy enterprise-grade solutions using Generative and Ag...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...