15286 Airflow Jobs - Page 12

Position: Data Engineer Experience: 3-5 Years Location: Mumbai(Andheri East) Employment Type: Full-time Job Description: We are seeking a skilled and detail-oriented Data Engineer to join our growing technology team. The ideal candidate will be responsible for designing, building, and maintaining reliable data pipelines while ensuring high performance and scalability. You will work closely with cross-functional teams to support data-driven decision-making and enhance our overall data infrastructure. Key Responsibilities: Build and maintain scalable data pipelines for ETL/ELT processes. Write clean, efficient, and optimized Python or Scala code for various data engineering tasks. Use SQL exte...

Since our launch in 2015, we’ve backed entrepreneurs with over £13bn in funding, helping create 56,000+ jobs and 34,000+ new homes across the UK and US and we’re not about to stop there. Our success is built on bold thinking and a refusal to accept “business as usual.” This role is part of our Product Operations team for Personal Savings (PS). Product Ops at OakNorth is about designing and running the processes, tooling, and controls that enable the business to scale with accuracy and speed. You’ll start by running today’s manual workflows, then lead the transition to automated, data-driven systems that improve reporting turnaround, rate-change governance, and partner satisfaction. If you’re...

We’re hiring a Senior QA Engineer who can own test automation across data/ETL pipelines, APIs, and web frontends. You’ll design robust QA strategies, build CI/CD-native test suites, drive performance/load testing, and turn system requirements into executable, measurable test plans. What will you do: ETL/Data Quality Automation Design and automate tests for ingestion, transformation, and loading (schema checks, constraints, null/dup checks, reconciliation, row/column-level validation, business rule assertions). Implement data contract and data lineage test coverage; ensure CDC/merge logic and dedup rules are verified. Add data checks to orchestration (e.g., Airflow/Prefect/Cloud Composer) and...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

Role & responsibilities Over 5 years of experience with machine learning, especially Recommender Systems. Strong understanding of data processing, ML infrastructure, and DevOps/MLOps. At least 2 years with cloud technologies (preferably AWS, AWS SageMaker). Proficient in Python, SQL, and data modeling; DBT is a plus. Familiarity with Spark, Airflow, PyTorch, Scikit-learn, Pandas, and Keras.

Job ID: Pyt-ETP-Pun-1125 Location: Pune Company Overview Bridgenext is a Global consulting company that provides technology-empowered business solutions for world-class organizations. Our Global Workforce of over 800 consultants provide best in class services to our clients to realize their digital transformation journey. Our clients span the emerging, mid-market and enterprise space. With multiple offices worldwide, we are uniquely positioned to deliver digital solutions to our clients leveraging Microsoft, Java and Open Source with a focus on Mobility, Cloud, Data Engineering and Intelligent Automation. Emtec’s singular mission is to create “Clients for Life” – long-term relationships that...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...

Hello All, Greetings from Tesslogs, We are seeking a skilled Big Data Engineer with strong experience in Google Cloud Platform (GCP) , Python , and data pipeline orchestration . The ideal candidate will be responsible for building, maintaining, and optimizing scalable data pipelines and analytics solutions using Airflow and BigQuery . Job Title: GCP Big Data Engineer Location: Gurugram Experience: 4–8 Years Employment Type: Full-time Key Responsibilities Design and develop scalable data pipelines on GCP using Cloud Composer (Airflow), Dataflow, and other GCP services. Build and automate ETL/ELT workflows using Python and Airflow DAGs. Develop and optimize BigQuery datasets , queries, stored ...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? Responsible for contacting clients with overdue accounts to secure the settlement of the account. Also they do preventive...

Data Engineers design and build data systems and pipelines. Responsibilities include developing data processing workflows, optimizing data storage, and ensuring data accuracy. Design and deployment of analytical models and dashboards to uncover insights, predict trends and inform decision-making. Collaborate and provide data support to transformation initiatives, ensuring alignment with business objectives and digital innovation goals. Oversee technical designs for data models, ETL and visualization tools (like BigQuery, Airflow, SSIS, Tableau, SSRS). Ensure that all the analytical code and data pipelines adhere to best practices for readability, maintainability and efficiency using clean co...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? Responsible for contacting clients with overdue accounts to secure the settlement of the account. Also they do preventive...

Role & responsibilities 5+ years of experience in data engineering roles. • Proficiency in programming languages such as Scala or Python, or Java. • Expertise in distributed data processing frameworks like Apache Spark or Flink. • Strong knowledge of cloud platforms (e.g., AWS) and cloud-native data platforms like Databricks or Redshift. • Solid understanding of SQL, schema design, dimensional modeling, and ETL best practices. • Experience with workflow orchestration tools such as Apache Airflow. • Familiarity with CI/CD pipelines, preferably GitHub Actions. • Strong technical documentation and issue-tracking skills. • Demonstrated ability to adopt and integrate new technologies.

Position Summary We are looking for a highly skilled and experienced Data Engineering Manager to lead our data engineering team. The ideal candidate will possess a strong technical background, strong project management abilities, and excellent client handling/stakeholder management skills. This role requires a strategic thinker who can drive the design, development and implementation of data solutions that meet our clients’ needs while ensuring the highest standards of quality and efficiency. Job Responsibilities Technology Leadership – Lead guide the team independently or with little support to design, implement deliver complex cloud-based data engineering / data warehousing project assignm...

Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie Big Data Processing: Hadoop, Kafka, Spark & Spark Structured Streaming. Experience on SOLID & DRY principles with Good Software Architecture & Design implementation experience ¢ Advanced Scala experience (e.g. Functional Programming, using Case classes, Complex Data Structures & Algorithms) ¢ Proficient in developing automated frameworks for unit & integration testing. ¢ Proficient with Kubernetes, Docker, Helm and related container technologies. ¢ Proficient in deploying and managing Spark workloads on Kubernetes clusters. ¢

About Position: Data Engineer with extensive experience in leading large-scale data migration projects, specializing in PySpark and Databricks. Proven ability to design and optimize high-performance data pipelines, refactor legacy ETL processes, and ensure data integrity across modern cloud platforms. Adept at collaborating with architects and guiding engineering teams to align solutions with enterprise data strategies. Role: Senior Data Engineer Location: Pune Experience: 6 to 9 Years Job Type: Full Time Employment What You'll Do: Design and optimise PySpark pipelines for large-scale data migration. Refactor stored procedures and ETL logic from Oracle to Databricks SQL. Implement validation...

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We’re also a mission-driven company. We cultivate a culture that gives every CrowdStriker both the flexibility and autonomy to own their...