15348 Airflow Jobs - Page 15

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

Hello All, Greetings from Tesslogs, We are seeking a skilled Big Data Engineer with strong experience in Google Cloud Platform (GCP) , Python , and data pipeline orchestration . The ideal candidate will be responsible for building, maintaining, and optimizing scalable data pipelines and analytics solutions using Airflow and BigQuery . Job Title: GCP Big Data Engineer Location: Gurugram Experience: 4–8 Years Employment Type: Full-time Key Responsibilities Design and develop scalable data pipelines on GCP using Cloud Composer (Airflow), Dataflow, and other GCP services. Build and automate ETL/ELT workflows using Python and Airflow DAGs. Develop and optimize BigQuery datasets , queries, stored ...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? Responsible for contacting clients with overdue accounts to secure the settlement of the account. Also they do preventive...

Data Engineers design and build data systems and pipelines. Responsibilities include developing data processing workflows, optimizing data storage, and ensuring data accuracy. Design and deployment of analytical models and dashboards to uncover insights, predict trends and inform decision-making. Collaborate and provide data support to transformation initiatives, ensuring alignment with business objectives and digital innovation goals. Oversee technical designs for data models, ETL and visualization tools (like BigQuery, Airflow, SSIS, Tableau, SSRS). Ensure that all the analytical code and data pipelines adhere to best practices for readability, maintainability and efficiency using clean co...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? Responsible for contacting clients with overdue accounts to secure the settlement of the account. Also they do preventive...

Role & responsibilities 5+ years of experience in data engineering roles. • Proficiency in programming languages such as Scala or Python, or Java. • Expertise in distributed data processing frameworks like Apache Spark or Flink. • Strong knowledge of cloud platforms (e.g., AWS) and cloud-native data platforms like Databricks or Redshift. • Solid understanding of SQL, schema design, dimensional modeling, and ETL best practices. • Experience with workflow orchestration tools such as Apache Airflow. • Familiarity with CI/CD pipelines, preferably GitHub Actions. • Strong technical documentation and issue-tracking skills. • Demonstrated ability to adopt and integrate new technologies.

Position Summary We are looking for a highly skilled and experienced Data Engineering Manager to lead our data engineering team. The ideal candidate will possess a strong technical background, strong project management abilities, and excellent client handling/stakeholder management skills. This role requires a strategic thinker who can drive the design, development and implementation of data solutions that meet our clients’ needs while ensuring the highest standards of quality and efficiency. Job Responsibilities Technology Leadership – Lead guide the team independently or with little support to design, implement deliver complex cloud-based data engineering / data warehousing project assignm...

Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie Big Data Processing: Hadoop, Kafka, Spark & Spark Structured Streaming. Experience on SOLID & DRY principles with Good Software Architecture & Design implementation experience ¢ Advanced Scala experience (e.g. Functional Programming, using Case classes, Complex Data Structures & Algorithms) ¢ Proficient in developing automated frameworks for unit & integration testing. ¢ Proficient with Kubernetes, Docker, Helm and related container technologies. ¢ Proficient in deploying and managing Spark workloads on Kubernetes clusters. ¢

About Position: Data Engineer with extensive experience in leading large-scale data migration projects, specializing in PySpark and Databricks. Proven ability to design and optimize high-performance data pipelines, refactor legacy ETL processes, and ensure data integrity across modern cloud platforms. Adept at collaborating with architects and guiding engineering teams to align solutions with enterprise data strategies. Role: Senior Data Engineer Location: Pune Experience: 6 to 9 Years Job Type: Full Time Employment What You'll Do: Design and optimise PySpark pipelines for large-scale data migration. Refactor stored procedures and ETL logic from Oracle to Databricks SQL. Implement validation...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

Job Title - Retail Specialized Data Scientist – Level 9 – SnC GN Data & AI Management Level: 09 - Consultant Location: Bangalore / Gurgaon / Mumbai / Chennai / Pune / Hyderabad / Kolkata Must have skills: A solid understanding of retail industry dynamics, including key performance indicators (KPIs) such as sales trends, customer segmentation, inventory turnover, and promotions. Strong ability to communicate complex data insights to non-technical stakeholders, including senior management, marketing, and operational teams. Meticulous in ensuring data quality, accuracy, and consistency when handling large, complex datasets. Gather and clean data from various retail sources, such as sales transa...

Position Overview We are seeking an experienced Senior Database Administrator (DBA) to own and optimize the performance, security, and scalability of our data platforms across PostgreSQL, Cloud SQL, and ClickHouse environments. The ideal candidate will bring hands-on expertise in replication, encryption, query optimization, automation, and cloud-native database management — working closely with Engineering and Operations teams to ensure seamless database performance, secure data handling, and efficient production migrations. This role is critical in designing and maintaining high-performing, resilient database ecosystems that power our fintech and data-driven products. Key Responsibilities D...

About Position: We are looking for a highly skilled DevOps Engineer with proven experience in large-scale enterprise environments. The ideal candidate will have a strong understanding of DevOps principles, hands-on expertise in Azure Cloud, CI/CD automation, Kubernetes, Docker, and Azure DevOps, along with proficiency in Infrastructure as Code (Terraform) and workflow orchestration tools like Airflow and Azure Data Factory. Role: DevOps Engineer Location: All Persistent Location Experience: 4 to 8 Years Job Type: Full Time Employment What You'll Do: Proven experience as a DevOps Engineer within a large-scale enterprise. A great understanding of DevOps with proven and practical experience. As...

Job Title: Data Engineer Experience Required: 3+ Years Location: Base Location – Doha, Qatar Must Have : Python, ETL, GCP, Pyspark The candidate should possess a valid driver's license and passport. About the Role- We are looking for a skilled Azure Data Engineer with hands-on experience in Google Cloud Platform (GCP) to design, build, and optimize data pipelines. The ideal candidate will have expertise in data ingestion, validation, and verification to ensure the reliability, scalability, and quality of data across multiple systems. Key Responsibilities- Design, develop, and maintain data pipelines and ETL processes on GCP. Implement robust data ingestion frameworks from various sources (ba...

Position Summary... Demonstrates up-to-date expertise and applies this to the development, execution, and improvement of action plans by providing expert advice and guidance to others in the application of information and best practices; supporting and aligning efforts to meet customer and business needs; and building commitment for perspectives and rationales. Provides and supports the implementation of business solutions by building relationships and partnerships with key stakeholders; identifying business needs; determining and carrying out necessary processes and practices; monitoring progress and results; recognizing and capitalizing on improvement opportunities; and adapting to competi...

Role & responsibilities Database & Analytics Big Data Developer - Hadoop / HDFS Resource should have expertise in all- Bigdata Spark ETL/Python , Airflow, DBT, & ADO Devops experience.

Some careers shine brighter than others. If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to join our team i...

3+ years experienced engineer who has worked on GCP environment and its relevant tools/services. (Big Query, Data Proc, Data flow, Cloud Storage, Terraform, Tekton , Cloudrun , Cloud scheduler, Astronomer/Airflow, Pub/sub, Kafka, Cloud Spanner streaming etc) 1 or 2 + years of strong experience in Python development (Object oriented/Functional Programming, Pandas, Pyspark etc) 1 or 2 + years of strong experience in SQL Language (CTEs, Window functions, Aggregate functions etc). Keywords : - airflow,kafka,pyspark,pandas,cloud storage,python development,terraform,cloud spanner,python,java,spark,hadoop,aws,hive,big data,mapreduce,javascript,mysql,postgresql,oracle,sql server,oops,gcp,bigquery,da...

Job Title: Founding Data Engineer (ETL & Web Integration)- 6 Months Contract About Us At GoDiverse, we're on a mission to build a fairer and more efficient economy for small businesses. We are an early-stage tech company building an AI-powered platform to help enterprises discover and engage with innovative Small and Medium-sized Enterprises (SMEs). Our goal is to level the playing field, ensuring that smaller, dynamic companies get fair access to large-scale commercial opportunities. The Role We are looking for our Founding Data Engineer to build and own the entire data acquisition and preparation lifecycle for our platform. This is a hardcore, hands-on engineering role with no project mana...

Role Overview: An AWS SME with a Data Science Background is responsible for leveraging Amazon Web Services (AWS) to design, implement, and manage data-driven solutions. This role involves a combination of cloud computing expertise and data science skills to optimize and innovate business processes. Key Responsibilities: Data Analysis and Modelling: Analyzing large datasets to derive actionable insights and building predictive models using AWS services like SageMaker, Bedrock, Textract etc. Cloud Infrastructure Management: Designing, deploying, and maintaining scalable cloud infrastructure on AWS to support data science workflows. Machine Learning Implementation: Developing and deploying mach...

Role: MLOps Engineer Required Experience: 3 - 5 years Job Location: : Gurugram Job Description: Must Have Skills: Cloud development (Capable),ML Ops - MLE Key Responsibilities: Collaborating with data scientists to understand model requirements, gather data, and refine models. Responsible for the lifecycle of models, once deployed in production environments, through monitoring performance and model evolution. Solve technical problems related to data processing, model building, and deployment. Building and training machine learning models, including selecting datasets, performing statistical analysis, and refining models. Deploy machine learning models to production environments, ensuring tha...

MLOpsWe are looking for a highly skilled Analytics & Data Engineering professional with a strong background in Machine Learning, MLOps, and DevOps. The ideal candidate will have experience designing and implementing scalable data and analytics pipelines, enabling production-grade ML systems, and supporting agent-based development leveraging MCP/OpenAPI to MCP wrapper and A2A protocols. This role combines hands-on technical work with solution design, and will require close collaboration with data scientists, product teams, and engineering stakeholders. ;Key ResponsibilitiesDesign, build, and maintain scalable data pipelines and ETL/ELT processes for analytics and ML workloads.Implement MLOps ...

Data Engineer ;Job Title: Data EngineerLocation: RemoteExperience: 4-7 years ;Job Summary: The Data Engineer is responsible for designing and building data models and pipelines on GCP, Azure, Snowflake, BigQuery, utilizing SQL & Python expertise and following coding best practices. ;Key Responsibilities:Design and build data models and pipelines on GCP BigQuery, Azure, SnowflakeDevelop and execute SQL queries (DML) to extract insights from data.Collaborate with stakeholders to understand data requirements and provide solutions.Ensure data quality and integrity by following data modeling principles and coding best practices.Design, build, and maintain robust, scalable ETL/ELT pipelines using ...

Data Engineer ;Job Title: Data EngineerLocation: RemoteExperience: 4-7 years ;Job Summary: The Data Engineer is responsible for designing and building data models and pipelines on GCP, Azure, Snowflake, BigQuery, utilizing SQL & Python expertise and following coding best practices. ;Key Responsibilities:Design and build data models and pipelines on GCP BigQuery, Azure, SnowflakeDevelop and execute SQL queries (DML) to extract insights from data.Collaborate with stakeholders to understand data requirements and provide solutions.Ensure data quality and integrity by following data modeling principles and coding best practices.Design, build, and maintain robust, scalable ETL/ELT pipelines using ...

Some careers shine brighter than others. If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organizations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realize their ambitions. We are currently seeking an experienced professional to join our team i...