682 Apache Airflow Jobs - Page 9

Role & responsibilities Job Description: Job Summary: We are looking for a skilled Data Engineer to join our data team. The ideal candidate will have hands-on experience in building robust data pipelines using Airflow, managing data warehouses in Snowflake and Redshift, and working with cloud platforms such as AWS and Azure (Databricks, Glue). You will work closely with Data Scientists, Analysts, and Engineers to ensure data availability, quality, and reliability across the organization. Key Responsibilities: l Design, develop, and maintain scalable ETL/ELT pipelines using Apache Airflow, AWS Glue, and Databricks. l Integrate data from multiple sources into Snowflake and Redshift data wareho...

As an AI Automation Engineer, your role will involve designing, developing, and implementing intelligent automation solutions using AI, machine learning, and automation tools. You will collaborate with cross-functional teams to integrate AI-powered automation into enterprise systems. Key Responsibilities: - Design and develop AI-driven automation solutions to optimize workflows and business operations. - Implement machine learning models and AI algorithms to enhance process automation. - Develop and maintain automation scripts, bots, and AI-based workflows. - Integrate AI automation with existing systems, APIs, and databases. - Utilize RPA tools such as UiPath, Automation Anywhere, or Blue P...

As the Test Automation Engineering Lead at Barclays, you will play a crucial role in spearheading the evolution of the digital landscape, driving innovation, and ensuring unparalleled customer experiences. Your responsibilities will include: - Developing and implementing comprehensive test plans and strategies to validate software functionality and ensure compliance with established quality standards. - Creating and executing automated test scripts using tools like Selenium, PyTest, Robot Framework, Apache Airflow, dbt, or similar to facilitate early detection of defects and quality issues. - Collaborating with cross-functional teams to analyze requirements, participate in design discussions...

Role Overview: Bridgenext is a global digital consultancy focused on helping clients innovate intentionally and achieve their digital goals by developing digital products, experiences, and solutions tailored to real people's needs. As an Engineer with hands-on Big Data experience at Bridgenext, you will have the opportunity to work on internal and customer-based projects, ensuring high-quality code and providing optimal solutions to meet clients" current and future needs. Key Responsibilities: - Possessing 3-4 years of overall experience - Demonstrating strong programming skills in Python/Java - Having 3+ years of experience with distributed systems and frameworks, especially Spark - Working...

As a Sr/Lead Data Engineer at Ccube in Indore, MP (Hybrid Model), you will have the opportunity to work with large and complex sets of data to meet both non-functional and functional business requirements. Your key responsibilities will include: - Gathering and assembling data using skills such as SQL, Python, R, Data Modeling, Data Warehousing, and AWS (S3, Athena) - Creating new data pipelines or enhancing existing pipelines with tools like ETL Tools (e.g., Apache NiFi, Talend), Python (Pandas, PySpark), AWS Glue, JSON, XML, YAML - Identifying, designing, and implementing process improvements with tools like Apache Airflow, Terraform, Kubernetes, AWS Lambda, CI/CD pipelines, Docker - Build...

Role Overview: At Techjays, we are on a mission to empower businesses worldwide by building AI solutions that drive industry transformation. We are seeking a skilled Data Analytics Engineer with 3 to 8 years of experience to join our global team in Coimbatore. As a Data Analytics Engineer at Techjays, you will play a crucial role in designing and delivering scalable, data-driven solutions that support real-time decision-making and deep business insights. Your primary focus will be on developing and maintaining ETL/ELT data pipelines, collaborating with various teams, designing interactive dashboards, and ensuring high reliability of data pipelines. If you are passionate about using tools lik...

As a Data Engineering Lead at Metyis, you will play a crucial role in independently leading and managing the execution of data engineering projects. You will be responsible for engineering complete technical solutions to solve concrete business challenges in areas such as digital marketing, eCommerce, Business Intelligence, and self-service analytics. Your key responsibilities will include: - Collecting functional and non-functional requirements, considering technical environments, business constraints, and enterprise organizations. - Supporting clients in executing their Big Data strategies by designing and building operational data platforms, including ETL pipelines, data anonymization pip...

As a Principal Software Engineer at People.AI, your primary role will involve leading the development of innovative ML systems on an enterprise scale. You will be responsible for implementing cutting-edge Generative AI technologies and collaborating with a diverse team to shape the technical direction of products and systems. Your key responsibilities will include: - Working in an Agile environment as part of a scrum team - Designing end-to-end system architecture for data-intensive applications - Developing and deploying production-grade real-time/batch machine learning services - Collaborating with applied researchers and data scientists to improve ML models and refine MLOps practice - Sup...

As a Python Web Scraper at HiLabs in Bengaluru, Karnataka, you will play a crucial role in designing and building scalable web scraping solutions using Python/PySpark. Your responsibilities will include developing enterprise-grade scraping services, working with structured and unstructured data, implementing data validation processes, and collaborating with cross-functional teams to ensure smooth data flow between systems. Your technical expertise in Python, web scraping tools and libraries, PySpark, cloud-based platforms, and databases will be essential for optimizing data workflows and ensuring data accuracy, consistency, and availability. **Responsibilities:** - Design and build scalable,...

As a Junior Python Engineer for A Single Platform for Loans | Securities | MSRs | Private Credit, you will be responsible for developing and maintaining Python-based applications and microservices. You will work on building and supporting Airflow DAGs for workflow orchestration and data automation, collaborate with senior engineers on designing scalable solutions, and utilize AI-assisted development tools like Cursor and GitHub Copilot. Your role will involve writing clean, testable, and maintainable code, debugging existing applications, and contributing to code reviews and team knowledge sharing. **Key Responsibilities:** - Develop and maintain Python-based applications and microservices. ...

Role Overview: As an AI Data Engineer, your primary responsibility will be to design, develop, and maintain data pipelines and infrastructure crucial for AI and machine learning projects. You will play a vital role in ensuring that models are trained on high-quality, well-prepared data and that data flows efficiently from diverse sources into AI and GenAI applications. This full-time on-site job is based at our office in Infopark, Kochi. Key Responsibilities: - Build, test, and maintain scalable data pipelines for AI and machine learning workflows. - Develop and manage architected data solutions such as warehouses, lakes, and streaming platforms to support generative and predictive AI use ca...

As a Software Engineer III at JPMorgan Chase within the Corporate Data Services, you will be a crucial part of an agile team that designs and delivers trusted market-leading technology products. Your role involves executing software solutions, design, development, and technical troubleshooting with the ability to think innovatively. You will be accountable for creating secure and high-quality production code, maintaining algorithms, and producing architecture and design artifacts for complex applications. Additionally, you will gather, analyze, and synthesize data sets to drive continuous improvement and contribute to software engineering communities of practice. Key Responsibilities: - Exec...

As a Machine Learning Engineer, you will be responsible for designing, developing, deploying, and maintaining scalable ML solutions in a cloud-native environment. Your key responsibilities will include: - Designing and implementing machine learning models and pipelines using AWS SageMaker and related services. - Developing and maintaining robust data pipelines for training and inference workflows. - Collaborating with data scientists, engineers, and product teams to translate business requirements into ML solutions. - Implementing MLOps best practices including CI/CD for ML, model versioning, monitoring, and retraining strategies. - Optimizing model performance and ensuring scalability and r...

Role: ETL Data engineer/ETL developer Experience: 3.9 Years - 9 Years Location: Chennai, Guindey,WFO Notice Period: Only Immediate Joiners SKills: Mandatory skills: 1.ETL 2.AWS,Lambda,SNS 3.Python,Pyspark 4.Apache airflow Interested candidates can share their updated to CV to pravallika@wrootsglobal.in

8+ years of experience building and scaling data platforms, data architecture, data modeling, and data warehousing. 3+ Years of experience working as an Architect, designing enterprise data solutions. Required Candidate profile Exp with cloud data technologies including Azure, Python Scripting, Apache Airflow and Databricks Exp in SQL and Data Modelling tools & concepts. Exp with data virtualization and data mesh arch.

As a skilled Data Engineer at our company, your role will involve a dual focus on infrastructure maintenance and seamless onboarding of data views. You will be instrumental in ensuring stable data platform operations and facilitating efficient data integration, particularly amidst complex upstream changes and fiscal year transitions. Key Responsibilities: - Perform infrastructure maintenance tasks such as Azure subscription management, Azure Infrastructure and Platform Operations, ETL pipeline monitoring, source path validation and updates, and proactive issue identification and resolution. - Manage data onboarding activities by integrating new data sources and views, making adjustments for ...

Position Requirements: We are looking for a Technical Lead Data Engineering with strong experience in Infrastructure & ETL development. This requires experience in setting up environments, tool installations, and cloud services to support ongoing development initiatives Roles & Responsibilities: Design and manage scalable data architectures that meet business needs and performance requirements. Lead the implementation of data storage solutions, such as data warehouses and data lakes, across hybrid and cloud-based environments (AWS, Azure, or GCP) Develop and enforce data governance, quality, and security standards to protect sensitive data and ensure compliance. Develop and optimize complex ...

As a Data Platform Engineer, your role will involve designing, building, and maintaining scalable data infrastructure using AWS cloud services. You will utilize your expertise in Python, PySpark, EMR, and Apache Airflow to develop robust data pipelines and analytics solutions that drive business insights. Key Responsibilities: - Design and implement scalable data pipelines using Apache Airflow - Build and optimize AWS EMR clusters for big data processing - Develop data processing applications using Python and PySpark - Create ETL workflows for data ingestion and transformation - Monitor and troubleshoot data platform performance - Collaborate with data scientists and analysts on data require...

As an experienced professional with 3+ years of experience in building and operating production-grade applications and services across the stack, you will be responsible for the following key responsibilities: - Demonstrating strong programming skills in Python and/or Scala and SQL to write modular, testable, and well-documented code for batch and streaming workloads. - Hands-on experience with modern data engineering stacks including: - Distributed processing using Apache Spark (Databricks preferred), PySpark/Scala. - Orchestration with Azure Data Factory or Apache Airflow; implementing event-driven patterns with Azure Functions/Logic Apps. - Working with storage & formats such as Delta Lak...

Onsite Opportunity Sr. Data Engineer / Lead Data Engineer Client: MNC Experience:8+years Location: Ras Al-Khaimah, UAE Job Mode: Permanent Salary Range: 15K AED to 20K AED Note: Selected candidates must hold a valid passport. Selected candidates should be open to travel to Ras Al-Khaimah, UAE. Candidates must possess strong communication skills. Key skills: Python, Data Engineer, Databricks, SQL, Shell Scripting, Snowflake, PySpark, dataframes, Azure Data Factory, Apache Airflow, and Databricks workflows, FastAPI, Azure etc.. Job Description: We are looking for a skilled Full Stack Developer with hands-on experience in building scalable web applications using Python (FastAPI/Flask/Django). Y...

Project Role Description: As data engineer the person will be responsible for data transformation & Migration in Amazon RedShift using Apache Airflow, Data Build Tool and Cosmos. Work location: Pune/Remote. Graduate or Post-Graduate in Computer Science/ Information Technology/Engineering. Joining Timelines: Immediate joiner Only . Job Requirements: Must Have Skills: 9 to 14 years IT Experience in data transformation in Amazon RedShift- Datawarehouse using Apache Airflow, Data Build Tool (DBT) and Azure DevOPS . Hands-on experience working in complex data warehouse implementations. Expert in Advance SQL. Expert in understanding Data Models is Must. Hands on experience parsing responses genera...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...