685 Apache Airflow Jobs - Page 25

As a Data Engineer II in the Deprecation Accelerator scope, you will be responsible for designing and maintaining scalable data pipelines both on-premises and on the cloud. Your role will involve understanding input and output data sources, managing upstream and downstream dependencies, and ensuring data quality. A crucial aspect of your job will be to focus on deprecating migrated workflows and migrating workflows to new systems when necessary. The ideal candidate will have expertise in tools like Git, Apache Airflow, Apache Spark, SQL, data migration, and data validation. Your key responsibilities will include: Workflow Deprecation: - Planning and executing the deprecation of migrated work...

You should have a minimum of 6 years of experience in the technical field and possess the following skills: Python, Spark SQL, PySpark, Apache Airflow, DBT, Snowflake, CI/CD, Git, GitHub, and AWS. Your role will involve understanding the existing code base in AWS services and SQL, and converting it to a tech stack primarily using Airflow, Iceberg, Python, and SQL. Your responsibilities will include designing and building data models to support business requirements, developing and maintaining data ingestion and processing systems, implementing data storage solutions, ensuring data consistency and accuracy through validation and cleansing techniques, and collaborating with cross-functional te...

As a Data Engineer at Blis, you will be part of a globally recognized and award-winning team that specializes in big data analytics and advertising. We collaborate with iconic brands like McDonald's, Samsung, and Mercedes Benz, providing precise audience insights to help them target their ideal customers effectively. Upholding ethical data practices and privacy rights is at the core of our operations, and we are committed to ensuring outstanding performance and reliability in all our systems. Working at Blis means being part of an international company with a diverse culture, spanning across four continents and comprising over 300 team members. Headquartered in the UK, we are financially suc...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

Job Summary: We are seeking a highly skilled Data Engineer to design, develop, and maintain robust data pipelines and architectures. The ideal candidate will transform raw, complex datasets into clean, structured, and scalable formats that enable analytics, reporting, and business intelligence across the organization. This role requires strong collaboration with data scientists, analysts, and cross-functional teams to ensure timely and accurate data availability and system performance. Key Responsibilities Design and implement scalable data pipelines to support real-time and batch processing. Develop and maintain ETL/ELT processes that move, clean, and organize data from multiple sources. Bu...

About Logik Are you driven to innovate Are you energized by the excitement of building a high-growth startup with winning technology and proven product-market fit Are you looking to join a team of A-players who keep customers first and take their work but not themselves seriously Logik was founded in 2021 by the godfathers of CPQ our CEO Christopher Shutts and our Executive Chairman Godard Abel, who together co-founded BigMachines, the first-ever CPQ technology vendor, in the early 2000s. Today, were reimagining what CPQ can and should be with our composable, AI-enabled platform that provides advanced configuration, transaction management, guided selling, and more. Were a well-funded and fas...

Job Summary: We are seeking a highly skilled and motivated Machine Learning Engineer with a strong foundation in programming and machine learning, hands-on experience with AWS Machine Learning services (especially SageMaker), and a solid understanding of Data Engineering and MLOps practices. You will be responsible for designing, developing, deploying, and maintaining scalable ML solutions in a cloud-native environment. Key Responsibilities: • Design and implement machine learning models and pipelines using AWS SageMaker and related services. • Develop and maintain robust data pipelines for training and inference workflows. • Collaborate with data scientists, engineers, and product teams to ...

The ideal candidate for this position should have advanced proficiency in Python, with a solid understanding of inheritance and classes. Additionally, the candidate should be well-versed in EMR, Athena, Redshift, AWS Glue, IAM roles, CloudFormation (CFT is optional), Apache Airflow, Git, SQL, Py-Spark, Open Metadata, and Data Lakehouse. Experience with metadata management is highly desirable, particularly with AWS Services such as S3. The candidate should possess the following key skills: - Creation of ETL Pipelines - Deploying code in EMR - Querying in Athena - Creating Airflow Dags for scheduling ETL pipelines - Knowledge of AWS Lambda and ability to create Lambda functions This role is fo...

As a Technical Program Manager (TPM), you will play a crucial role in establishing a strong connection between the business and engineering departments. Your primary responsibility will involve working on intricate business constraints and translating them into essential product requirements and features. Your technical expertise will be instrumental in guiding the team from the initial project stages through to its successful launch within strict timelines. A key aspect of your role will be to demonstrate exceptional leadership skills, inspiring teams to strive for excellence and develop top-notch products. Specific requirements for this role include having a fundamental understanding of va...

Genpact is a global professional services and solutions firm focused on delivering outcomes that shape the future. With over 125,000 employees across 30+ countries, we are driven by curiosity, entrepreneurial agility, and the desire to create lasting value for our clients. Our purpose, the relentless pursuit of a world that works better for people, guides us as we serve and transform leading enterprises, including the Fortune Global 500. Our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI enable us to drive results and deliver innovative solutions. We are currently seeking applications for the role of Principal Consultant Software ...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

-Maintain and update our in-house Shiny dashboard (driven by R) -Build SQL data pipelines for merchants and business analytics team -Collaborate with merchants, sales, and other stakeholders to identify and implement solutions to business needs.

Must have skills required: GCP, support, Python Forbes Advisor is Looking for: Role Summary We are seeking a proactive and detail-oriented Data Support Engineer- to monitor production processes, manage incident tickets, and ensure seamless operations in our data platforms. The ideal candidate will have experience in Google Cloud Platform (GCP), Airflow, Python and SQL with a strong focus on enabling developer productivity and maintaining system reliability. Key Responsibilities: Production Monitoring: Monitor and ensure the smooth execution of production data pipelines and workflows. Identify and promptly address anomalies or failures in the production environment. Perform first-level invest...

Job Title: Business Intelligence Analyst (R Shiny Developer) Position Overview: We are seeking an entry-level Business Intelligence Analyst (R Shiny Developer) to join our business analytics team. In this high-impact role, you will be responsible for maintaining and enhancing our in-house merchandising Shiny dashboard, adding new functionality to automate processes and analytics for our merchants. Additionally, you will be responsible for supporting the business analytics team with their business intelligence and data engineering needs. This position requires collaboration across multiple departments and offers an excellent opportunity to make a significant contribution to our company's succ...

Design and implement automated test cases for ETL processes and data pipelines Perform data validation, data transformation, and reconciliation testing Write and execute complex SQL queries to validate source-to-target data mappings Required Candidate profile Work closely ETL developers, business analysts, QA teams Log, track, and report defects using tools like JIRA, HP ALM, or TestRail Support regression testing, UAT, and performance testing for ETL jobs Perks and benefits Perks and Benefits

We are seeking a highly skilled Senior Data Engineer to join our dynamic team in Bangalore. You will design, develop, and maintain scalable data ingestion frameworks and ELT pipelines using tools such as DBT, Apache Airflow, and Prefect. The ideal candidate will have deep technical expertise in cloud platforms (especially AWS), data architecture, and orchestration tools. You will work with modern cloud data warehouses like Snowflake, Redshift, or Databricks and integrate pipelines with AWS services such as S3, Lambda, Step Functions, and Glue. A strong background in SQL, scripting, and CI/CD practices is essential. Experience with data systems in manufacturing is a plus.

We are looking for energetic, self-motivated and exceptional Data engineers to work on extraordinary enterprise products based on AI and Big Data engineering leveraging AWS/Databricks tech stack. He/she will work with a star team of Architects, Data Scientists/AI Specialists, Data Engineers and Integration. Skills and Qualifications: 5+ years of experience in DWH/ETL Domain; Databricks/AWS tech stack 2+ years of experience in building data pipelines with Databricks/ PySpark /SQL Experience in writing and interpreting SQL queries, designing data models and data standards. Experience in SQL Server databases, Oracle and/or cloud databases. Experience in data warehousing and data mart, Star and ...

Experience : 3+ years Shift : (GMT+05:30) Asia/Kolkata (IST) Must have Skills required: Python, Django, Fast API, Prompt Engineering, Apache Airflow, Mongo DB As a Python Developer at Infrrd, your job responsibilities will include: Create python scripts and services. Document inside code and designs. Ensure the deliverables are of the highest quality in terms of functional and technical aspects. Ensure the in sprint defects are closed with the highest quality and any production defects are being taken care of for a specific duration post release. Work along with the PM, Architect and the BA to manage the deliverables from technical and functional aspects Work along with DevOps, and other tea...

We are looking for energetic, self-motivated and exceptional Data engineer to work on extraordinary enterprise products based on AI and Big Data engineering leveraging AWS/Databricks tech stack. He/she will work with star team of Architects, Data Scientists/AI Specialists, Data Engineers and Integration. Skills and Qualifications: 5+ years of experience in DWH/ETL Domain; Databricks/AWS tech stack 2+ years of experience in building data pipelines with Databricks/ PySpark /SQL Experience in writing and interpreting SQL queries, designing data models and data standards. Experience in SQL Server databases, Oracle and/or cloud databases. Experience in data warehousing and data mart, Star and Sno...

We are looking for energetic, self-motivated and exceptional Data engineer to work on extraordinary enterprise products based on AI and Big Data engineering leveraging AWS/Databricks tech stack. He/she will work with star team of Architects, Data Scientists/AI Specialists, Data Engineers and Integration. Skills and Qualifications: 5+ years of experience in DWH/ETL Domain; Databricks/AWS tech stack 2+ years of experience in building data pipelines with Databricks/ PySpark /SQL Experience in writing and interpreting SQL queries, designing data models and data standards. Experience in SQL Server databases, Oracle and/or cloud databases. Experience in data warehousing and data mart, Star and Sno...

5 to 7 years of experience in data engineering Architect and maintain scalable, secure, and reliable data platforms and pipelines Design and implement data lake/data warehouse solutions such as Redshift, BigQuery, Snowflake, or Delta Lake Build real-time and batch data pipelines using tools like Apache Airflow, Kafka, Spark, and DBT Ensure data governance, lineage, quality, and observability

Notice period: Immediate 15days Profile source: Tamil Nadu Timings: 1:00pm 10:00pm (IST) Work Mode: WFO (Mon-Fri) We are seeking a detail-oriented and highly motivated Data Engineer to join our growing Data & Analytics team. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure that power insights across the organization. Youll work closely with data scientists, analysts, and engineers to ensure the integrity, accessibility, and scalability of our data systems. Key Responsibilities Responsibilities: Design, develop, and maintain scalable data pipelines and ETL Build and optimize data architecture to ensure data quality and Int...