974 Apache Airflow Jobs - Page 5

We are looking for a skilled Python developer with expertise in Airflow to join our team at IDESLABS PRIVATE LIMITED. The ideal candidate should have 6-10 years of experience. Roles and Responsibility Design, develop, and deploy scalable data pipelines using Apache Airflow. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop complex data transformations and scripts using Python. Troubleshoot and optimize existing data pipelines for improved efficiency. Ensure data quality and integrity by implementing robust testing and validation procedures. Participate in code reviews and contribute to the improvement of the overall codebase. Job Requirements St...

We are looking for a skilled Python developer with expertise in Airflow to join our team at IDESLABS PRIVATE LIMITED. The ideal candidate should have 6-10 years of experience. Roles and Responsibility Design, develop, and deploy scalable data pipelines using Apache Airflow. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop complex data transformations and scripts using Python. Troubleshoot and optimize existing data pipelines for improved efficiency. Ensure data quality and integrity by implementing robust testing and validation procedures. Participate in code reviews and contribute to the improvement of the overall codebase. Job Requirements St...

As a Senior Java Microservices Developer at Keyrus, you will be responsible for developing, maintaining, and optimizing Java 8+ backend services using Spring Boot and Hibernate/JPA. Your key responsibilities will include: - Building, deploying, and maintaining microservices within Kubernetes (AKS or equivalent) - Designing and implementing ETL pipelines using Apache Airflow, Spring Batch, or Apache Camel - Working with Snowflake to create and manage pipelines, connect and deploy databases, and perform query optimization - Integrating messaging solutions using Kafka or other enterprise messaging frameworks - Collaborating with cloud infrastructure teams to deploy and maintain services in Azur...

Role Overview: As a Data Engineer at ValGenesis, you will play a crucial role in designing, developing, and maintaining data ingestion, transformation, and orchestration pipelines. You will be responsible for building and optimizing data Lakehouse architectures using frameworks like Azure Synapse and Delta Lake. Collaborating with Data Scientists, you will prepare and serve ML-ready datasets for model training and inference. Ensuring scalability, performance, and observability of data workflows through effective monitoring and automation will be key to your role. Key Responsibilities: - Design, develop, and maintain data ingestion, transformation, and orchestration pipelines (batch and real-...

Job Description: As an EY BOT Data Engineer, you will be responsible for writing ETL (Extract / Transform / Load) processes, designing database systems, and developing tools for real-time and offline analytic processing. Your role will involve troubleshooting software and processes for data consistency and integrity, integrating large-scale data from various sources for business partners to generate insights and make decisions. You will translate business specifications into design specifications and code, writing complex programs, ad hoc queries, and reports. It is important that all code is well-structured, includes sufficient documentation, and is easy to maintain and reuse. Key Responsib...

As a Data Engineer at Rearc, you'll contribute to the technical excellence of the data engineering team. Your expertise in data architecture, ETL processes, and data modeling will help optimize data workflows for efficiency, scalability, and reliability. Working closely with cross-functional teams, you will design and implement robust data solutions that meet business objectives and adhere to best practices in data management. Building strong partnerships with technical teams and stakeholders will be essential as you support data-driven initiatives and contribute to their successful implementation. **Key Responsibilities:** - Collaborate with Colleagues: Work closely with colleagues to under...

As a Machine Learning Engineer at our company, you will play a crucial role in designing, developing, deploying, and maintaining scalable ML solutions in a cloud-native environment. Your strong programming skills and hands-on experience with AWS Machine Learning services, especially SageMaker, will be key in achieving success in this role. **Key Responsibilities:** - Design and implement machine learning models and pipelines using AWS SageMaker and related services. - Develop and maintain robust data pipelines for training and inference workflows. - Collaborate with data scientists, engineers, and product teams to translate business requirements into ML solutions. - Implement MLOps best prac...

You will be responsible for architecting and implementing scalable offline data pipelines for manufacturing systems, including AMHS, MES, SCADA, PLCs, vision systems, and sensor data. Your role will involve designing and optimizing ETL/ELT workflows using Python, Spark, SQL, and orchestration tools like Airflow to transform raw data into actionable insights. Additionally, you will lead database design and performance tuning across SQL and NoSQL systems, optimizing schema design, queries, and indexing strategies for manufacturing data. To enforce robust data governance, you will implement data quality checks, lineage tracking, access controls, security measures, and retention policies. You wi...

Python, PySpark, Apache Airflow & RestAPI Pyspark is mandatory. Hiting for AI product team.

Your Opportunity As an architect, you will get the opportunity to grow in the field of high-performance computing, complex system design and low-level optimizations for better cost of ownership. Roles and Responsibility As a Software Architect, you will be responsible for designing and implementing High performance computing software solutions for our organization. You will work closely with cross-functional teams, including software engineers, product managers, and business stakeholders, to understand requirements and translate them into architectural/software designs that meet business needs. You will be coding and developing quick prototypes to establish your design with real code and dat...

Summary As a Software tech lead, this role requires specialized depth and breadth of expertise in software architecture. The ideal candidate will interpret internal and external business issues to recommend best practices and solve complex problems. They will take a broad perspective to identify innovative solutions and work independently, receiving guidance only in the most complex situations. Additionally, the candidate may lead functional teams or projects, demonstrating strong leadership and collaboration skills. About Applied Applied Materials is the leader in materials engineering solutions used to produce virtually every new chip and advanced display in the world. Our expertise in mod...

Responsibilities: As a seasoned Software Expert, you will be responsible for designing and implementing scalable, reliable and maintainable software solutions for our organization. You will work closely with cross-functional teams, including software engineers, product managers, and business stakeholders, to understand requirements and translate them into architectural/software designs that meet business needs. You will be coding and developing quick prototypes to establish your design with real code and data. You will be a subject Matter expert to unblock software engineers. Conduct technical reviews and provide guidance to software engineers during the development process. Identify and mit...

You are a highly skilled Senior Backend Developer with expertise in building web frameworks using JavaScript/TypeScript and Node.js. You have hands-on experience with modern web frameworks like Koa and possess a strong knowledge of API design to support front-end applications. Your familiarity with backend services in Python, orchestration workflows with Airflow, and React-based front-end integrations is considered a plus. **Key Responsibilities:** - Design and develop scalable backend systems using JavaScript/TypeScript and Node.js. - Build robust web frameworks leveraging tools like Koa or similar frameworks. - Develop and maintain APIs to support React-based front-end applications. - Inte...

As a seasoned Solutions Architect with expertise in Snowflake, DBT, and Apache Airflow, your role will involve designing, architecting, and overseeing the implementation of scalable, secure, and high-performing data solutions. You will collaborate with stakeholders to understand data requirements, define best practices for data modeling and governance, guide and mentor data engineering teams, evaluate tools and frameworks, and ensure compliance with data security and regulatory requirements. Additionally, you will lead architecture reviews, data quality audits, and technical design sessions, while monitoring and troubleshooting performance issues related to Snowflake, Airflow DAGs, and DBT t...

Job Description: You will be responsible for writing ETL (Extract / Transform / Load) processes, designing database systems, and developing tools for real-time and offline analytic processing. Your role will involve troubleshooting software and processes to ensure data consistency and integrity. You will integrate large scale data from various sources to generate insights and facilitate decision-making for business partners. Additionally, you will translate business specifications into design specifications and code, writing complex programs, ad hoc queries, and reports. It will be your responsibility to ensure that all code is well structured, documented adequately, and easy to maintain and...

Role Overview: As the Head of Data Engineering at Spiro, you will be responsible for leading, designing, and scaling the next-generation data infrastructure. Your role will involve overseeing the development of robust, scalable data pipelines, architecting big data platforms, and ensuring the integrity and quality of the data that powers AI, analytics, and digital products. You will be a hands-on leader guiding a team of talented engineers and collaborating closely with analytics, software development, and product teams to drive business outcomes through data. This is a critical leadership position that will shape the future of data across various domains such as EV manufacturing, IoT telema...

Job Description: As a Tech Lead specializing in Snowflake (Data Engineering) at Nuaav, you will play a crucial role in leading the design, development, and optimization of data pipelines and Snowflake-based solutions. Your responsibilities will include architecting scalable Snowflake data models, guiding and mentoring a team of data engineers, collaborating with clients to design efficient data solutions, and ensuring high standards of code quality and performance tuning. Additionally, you will integrate Snowflake pipelines with Spark, Airflow, Kafka, and Azure cloud services to deliver robust and high-performance data architectures for enterprise environments. Key Responsibilities: - Lead t...

As a technical architect, you will be responsible for ensuring alignment between multiple development teams in data engineering and data science. You will design scalable, high-performance data processing solutions within Azure and Databricks, taking into account factors such as governance, security, and maintainability. Your role will involve driving the adoption of best practices in software development, including CI/CD, testing strategies, and cloud-native architecture. Additionally, you will work closely with Product Owners and Engineering Managers to translate business needs into technical solutions. - Act as a technical architect, ensuring alignment between multiple development teams -...

Role Overview: As an experienced Airflow Migration Developer, you will be responsible for leading the migration of job scheduling processes from Autosys to Apache Airflow. Your role will involve analyzing existing job definitions, designing and implementing equivalent Apache Airflow DAGs, integrating Airflow with downstream systems, and collaborating with cloud engineering and application teams for successful deployment in AWS environments. Your focus will be on ensuring a seamless transition, robust scheduling, and high availability in production environments. Key Responsibilities: - Analyze existing Autosys job definitions, dependencies, and workflows. - Design and implement equivalent Apa...

As a Cloud Associate in CloudOps, your role involves setting up and maintaining the monitoring infrastructure, ensuring the health, performance, and availability of systems, networks, applications, and services. You will collaborate with various teams and implement performance improvements for optimal system performance. Your responsibilities include incident detection and response, alerting and escalation, performance analysis, capacity planning, documentation and reporting, continuous improvement, as well as collaboration and communication. Key Responsibilities: - Set up and maintain monitoring infrastructure, including selecting and configuring monitoring tools, agents, and sensors. - Mon...

You will be responsible for providing advanced data solutions by using software to process, store, and serve data to others. You will test data quality and optimize data availability, ensuring that data pipelines are scalable, repeatable, and secure. Utilize a deep dive analytical skillset on a variety of internal and external data. **Key Responsibilities:** - Write ETL (Extract / Transform / Load) processes, design database systems, and develop tools for real-time and offline analytic processing. - Troubleshoot software and processes for data consistency and integrity. Integrate large-scale data from a variety of sources for business partners to generate insight and make decisions. - Transl...

As a Senior Backend Engineer at our company, your role will involve designing, developing, and maintaining backend applications using Python. You will collaborate with cross-functional teams to implement RESTful APIs and web services. Your responsibilities will also include optimizing database performance, developing and managing orchestration workflows, implementing CI/CD pipelines, and collaborating with DevOps teams for infrastructure management. Additionally, you will be expected to maintain high-quality documentation and adhere to version control practices. Key Responsibilities: - Design, develop, and maintain backend applications using Python. - Collaborate with cross-functional teams ...

Job Title : Python + AWS ( Airflow ) Candidate Specification : Any Graduate, 4+ years of Experience Job Description: Develop and maintain data pipelines and ETL/ELT workflows using Python and Apache Airflow. Build solutions using AWS services such as S3, Lambda, Glue, EC2, RDS, Redshift, Athena, Step Functions, and DynamoDB. Implement Airflow DAGs with best practices in scheduling, monitoring, error handling, and dependency management. Write optimized Python scripts for data ingestion, transformation, and integration. Design and maintain data models and ensure data quality, reliability, and consistency across pipelines. Work with DevOps teams to deploy Airflow workflows using CI/CD pipelines...

Key Responsibilities: Design, develop, and optimize complex PL/SQL procedures, packages, triggers, and functions . Write efficient Python scripts for data transformation, ETL automation, system integration, and reporting tasks. Analyze business requirements and translate them into technical specifications. Troubleshoot and resolve performance issues in PL/SQL queries and stored procedures. Ensure data integrity, quality, and availability across systems. Integrate data between systems using Python (e.g., REST APIs, flat files, database connectors). Support deployment, monitoring, and maintenance of applications in production. Collaborate with data engineers, analysts, and application develope...

As an Airflow Infrastructure & Automation Engineer at Qualcomm India Private Limited, you will play a crucial role in designing, scaling, and maintaining Apache Airflow workflow orchestration systems that are essential for automating critical processes in the CPU development lifecycle. Your responsibilities will include: - Manage Multi-Instance Airflow Deployments: You will be responsible for setting up and managing multiple Apache Airflow instances across different data centers to ensure high availability, security, and performance. - Custom Development: Develop custom Airflow Directed Acyclic Graphs (DAGs), operators, providers, and plugins to cater to specialized workflow requirements. - ...