10830 Airflow Jobs - Page 10

Responsibilities: Lead the design and development of scalable data pipelines using AWS (Glue, Redshift, Athena) Drive architecture decisions for data systems that support business analytics and application features Build end-to-end solutions, including backend APIs and, where applicable, simple UI components Orchestrate workflows using Airflow or equivalent tools, ensuring dependency management and error handling Ensure high standards of data quality through validation frameworks and monitoring Automate deployment and testing processes through CI/CD for data infrastructure Partner with cross-functional teams to understand data needs and deliver reliable, performant solutions Mentor junior en...

As a member of this team, the data engineer will be responsible for designing and expanding our existing data infrastructure, enabling easy access to data, supporting complex data analyses, and automating optimization workflows for business and marketing operations Essential Responsibilities: As a Senior Software Engineer, your responsibilities will include: Building, refining, tuning, and maintaining our real-time and batch data infrastructure Daily use technologies such as Python, Spark, Airflow, Snowflake, Hive, FastAPI, etc. Maintaining data quality and accuracy across production data systems Working with Data Analysts to develop ETL processes for analysis and reporting Working with Prod...

Essential Responsibilities: As a Senior Software Engineer, your responsibilities will include: Building, refining, tuning, and maintaining our real-time and batch data infrastructure Daily use technologies such as Python, Spark, Airflow, Snowflake, Hive, FastAPI, etc. Maintaining data quality and accuracy across production data systems Working with Data Analysts to develop ETL processes for analysis and reporting Working with Product Managers to design and build data products Working with our DevOps team to scale and optimize our data infrastructure Participate in architecture discussions, influence the road map, take ownership and responsibility over new projects Participating in on-call ro...

Essential Responsibilities: As a Senior Software Engineer, your responsibilities will include: Building, refining, tuning, and maintaining our real-time and batch data infrastructure Daily use technologies such as Python, Spark, Airflow, Snowflake, Hive, FastAPI, etc. Maintaining data quality and accuracy across production data systems Working with Data Analysts to develop ETL processes for analysis and reporting Working with Product Managers to design and build data products Working with our DevOps team to scale and optimize our data infrastructure Participate in architecture discussions, influence the road map, take ownership and responsibility over new projects Participating in on-call ro...

THE ROLE Zeta Marketing Platform (ZMP) is a machine learning/AI powered customer acquisition and CRM multi-tenant platform. The Sr backend developer will work on server-side APIs and services that enable a highly distributed event pipeline and a stack that gets tens of thousands of messages per second. As a Senior Software Engineer you will join provide technical leadership to the group responsible for architecting, developing, and owning the Zeta Marketing Platform. You will collaborate with Engineers, Product Managers and Executives across the organization to develop a roadmap and subsequent projects to build the next generation comprehensive, multichannel marketing solution that unifies a...

Lead Software Engineer Backend Were seeking a Lead Software Engineer to join one of our Data Layer teams. As the name implies, the Data Layer is at the core of all things data at Zeta. Our responsibilities include: Developing and maintaining the Zeta Identity Graph platform, which collects billions of behavioural, demographic, environmental, and transactional signals to power people-based marketing. Ingesting vast amounts of identity and event data from our customers and partners. Facilitating data transfers across systems. Ensuring the integrity and health of our datasets. And much more. As a member of this team, the data engineer will be responsible for designing and expanding our existing...

Responsibilities : ETL Development The CDP ETL & Database Engineer will be responsible for building pipelines to feed downstream data processes. They will be able to analyze data, interpret business requirements, and establish relationships between data sets. The ideal candidate will be familiar with different encoding formats and file layouts such as JSON and XML. I mplementations & Onboarding Will work with the team to onboard new clients onto the ZMP/CDP+ platform. The candidate will solidify business requirements, perform ETL file validation, establish users, perform complex aggregations, and syndicate data across platforms. The hands-on engineer will take a test-driven approach towards ...

Our Team Our team is developing a high-performance computing solution for low-latency and high throughput image processing and deep-learning workload that enables our Chip Manufacturing process control equipment to offer differentiated value to our customers. Your Opportunity As an architect, you will get the opportunity to grow in the field of high-performance computing, complex system design and low-level optimizations for better cost of ownership. Roles and Responsibility As a Software Architect, you will be responsible for designing and implementing High performance computing software solutions for our organization. You will work closely with cross-functional teams, including software en...

Director Data Science and AI Location: Mumbai or Bangalore About Traya Traya is a health-tech company that treats hair loss end-to-end through a personalized program combining dermatology, Ayurveda, nutrition, and behavioral science. We own the entire pathway assessment, regimen design, coaching, refills, and protocol adjustments powered by an adherence engine and outcomes data that improve results over time. As Traya scales into an omnichannel health platform (web, app, clinics, tele-coaching), data science is the engine powering personalization, retention, and efficiency . Why This Role Exists This is a hands-on builder role for a leader who wants to take models from notebook to production...

About The Role Project Role : AI / ML Engineer Project Role Description : Develops applications and systems that utilize AI tools, Cloud AI services, with proper cloud or on-prem application pipeline with production ready quality. Be able to apply GenAI models as part of the solution. Could also include but not limited to deep learning, neural networks, chatbots, image processing. Must have skills : Machine Learning Operations Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Machine Learning Engineer/MLOps Expert, you will engage in the operationalization of Machine Learning Models that leverage artifi...

Project Role : AI / ML Engineer Project Role Description : Develops applications and systems that utilize AI tools, Cloud AI services, with proper cloud or on-prem application pipeline with production ready quality. Be able to apply GenAI models as part of the solution. Could also include but not limited to deep learning, neural networks, chatbots, image processing. Must have skills : Machine Learning Operations Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Machine Learning Engineer/MLOps Expert, you will engage in the operationalization of Machine Learning Models that leverage artificial intelligen...

Location: Kochi / Bangalore (initially in Kochi) Role Summary We are seeking a Data Engineering Lead to architect, build, and optimize scalable data pipelines and data platforms that power critical business decisions. The ideal candidate will have deep expertise in PySpark, ETL frameworks, SQL Stored Procedures, and Data Modeling, along with a strong focus on performance tuning across large datasets. Key Responsibilities  Lead the design and development of scalable data pipelines and ETL workflows using PySpark and SQL  Develop and maintain Stored Procedures to support batch processes and data transformations  Design and implement robust data models (Star, Snowflake, and normalized models...

Location: Kochi / Bangalore (initially in Kochi) Role Summary We are looking for a hands-on AI/ML Engineer to design, build, and deploy production-grade machine learning systems. The role requires strong experience in both model development and MLOps practices, ensuring that ML solutions are scalable, maintainable, and performance optimized. Key Responsibilities Design and implement ML models for classification, regression, clustering, Recommendation System, Anomaly Detection, Neural Networks, NLP, Comp Vision Models Package models and deploy them into production environments using CI/CD workflows Develop and manage ML pipelines for data preprocessing, training, testing, and serving Moni...

Job Summary: We are looking for an experienced and highly skilled Senior Data Engineer to lead the design and development of our data infrastructure and pipelines. As a key member of the Data & Analytics team, you will play a pivotal role in scaling our data ecosystem, driving data engineering best practices, and mentoring junior engineers. This role is ideal for someone who thrives on solving complex data challenges and building systems that power business intelligence, analytics, and advanced data products. Key Responsibilities: Design and build robust, scalable, and secure data pipelines and models. Lead the complete lifecycle of ETL/ELT processes, encompassing data intake, transformation...

Project Role : AI / ML Engineer Project Role Description : Develops applications and systems that utilize AI tools, Cloud AI services, with proper cloud or on-prem application pipeline with production ready quality. Be able to apply GenAI models as part of the solution. Could also include but not limited to deep learning, neural networks, chatbots, image processing. Must have skills : Machine Learning Operations Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Machine Learning Engineer/MLOps Expert, you will engage in the operationalization of Machine Learning Models that leverage artificial intelligen...

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Spring Boot Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications are aligned with business objectives. You will also engage in problem-solving discussion...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Snowflake Data Warehouse, Data Engineering Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Snowflake Sr. Data Engineer, you will build scalable, secure, and high-performance Snowflake pipelines across multiple projects and teams. Your typical day will involve collaborating with Team Leads to ens...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Snowflake Data Warehouse Good to have skills : Data Engineering, Data Building Tool, SAP SD, Fivetran, Gen AIMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Snowflake Lead, you will build scalable, secure, and high-performance Snowflake pipelines across multiple projects and teams. Your typical day will involve collabo...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Snowflake Data Warehouse, Data Engineering Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Snowflake Sr. Data Engineer, you will build scalable, secure, and high-performance Snowflake pipelines across multiple projects and teams. Your typical day will involve collaborating with Team Leads to ens...

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Spring Boot Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications are aligned with business objectives. You will also engage in problem-solving discussion...

Design, develop, and manage our data infrastructure on AWS, with a focus on data warehousing solutions. Write efficient, complex SQL queries for data extraction, transformation, and loading. Utilize DBT for data modelling and transformation. Use Python for data engineering tasks, demonstrating strong work experience in this area. Implement scheduling tools like Airflow, Control M, or shell scripting to automate data processes and workflows. Participate in an Agile environment, adapting quickly to changing priorities and requirements. Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Proven expertise in AWS technologies, wit...

About the Role We are looking for a Software Development Engineer III specializing in Infrastructure and Data Engineering. You will design, build, and optimize scalable infrastructure and data pipelines that enable our product, analytics, and machine learning teams to deliver reliable insights and services. As a senior IC, you will work on cloud infrastructure, data modeling, and distributed systems while collaborating with product, backend, and analytics stakeholders. What You Will Do 1. Infrastructure Engineering a. Design, build, and maintain scalable cloud infrastructure (AWS/GCP/Azure). b. Implement Infrastructure-as-Code (IaC) using Terraform, CloudFormation, or similar tools. c. Ensur...

Overview GCP, Big Query, Composer, Python, Pyspark, Dataproc, Airflow, SQL Roles and Responsibility Design and implement data pipelines using Google Cloud Platform services. Develop and maintain large-scale data architectures and systems. Collaborate with cross-functional teams to identify and prioritize project requirements. Ensure data quality and integrity by implementing robust testing and validation procedures. Optimize data processing workflows for improved performance and efficiency. Troubleshoot and resolve complex technical issues related to data engineering. Job Requirements Strong understanding of Google Cloud Platform services and technologies. Experience with data modeling, desi...

About The Role : Role Purpose Data Analyst, Data Modeling, Data Pipeline, ETL Process, Tableau, SQL, Snowflake. Do Strong expertise in data modeling, data warehousing, and ETL processes. - Proficient in SQL and experience with data warehousing tools (e.g., Snowflake, Redshift, BigQuery) and ETL tools (e.g., Talend, Informatica, SSIS). - Demonstrated ability to lead and manage complex projects involving cross-functional teams. - Excellent analytical, problem-solving, and organizational skills. - Strong communication and leadership abilities, with a track record of mentoring and developing team members. - Experience with data visualization tools (e.g., Tableau, Power BI) is a plus. - Preferenc...

Job Description We are seeking a highly skilled Data Scientist along with strong Data Engineering expertise to join Advance Analytics and Modeling team. The ideal candidate will have a proven ability to work with diverse datasets, build machine learning models, and deploy scalable solutions. This role requires proficiency in Python, advanced analytical skills, and experience in developing machine learning models and dashboards in python. The candidate should have experience in deploying machine learning models in production using cloud platforms such as AWS or Azure and workflow orchestration tools like Airflow. Key Responsibilities Develop and deploy machine learning models for classificati...