15348 Airflow Jobs - Page 27

Description Job Title : Lead Data Engineer- contract Job Summary The Lead Data Engineer will provide technical expertise in analysis, design, development, rollout and maintenance of data integration initiatives. This role will contribute to implementation methodologies and best practices, as well as work on project teams to analyze, design, develop and deploy business intelligence / data integration solutions to support a variety of customer needs. This position oversees a team of Data Integration Consultants at various levels, ensuring their success on projects, goals, trainings and initiatives though mentoring and coaching. Provides technical expertise in needs identification, data modelli...

Expedia Group brands power global travel for everyone, everywhere. We design cutting-edge tech to make travel smoother and more memorable, and we create groundbreaking solutions for our partners. Our diverse, vibrant, and welcoming community is essential in driving our success. Why Join Us? To shape the future of travel, people must come first. Guided by our Values and Leadership Agreements, we foster an open culture where everyone belongs, differences are celebrated and know that when one of us wins, we all win. We provide a full benefits package, including exciting travel perks, generous time-off, parental leave, a flexible work model (with some pretty cool offices), and career development...

5 to 8 years of experience designing and building data pipelines using Apache Spark, Databricks or equivalent bigdata frameworks. Hands on expertise with streaming and messaging systems such as Apache Kafka (publish subscribe architecture), Confluent Cloud, RabbitMQ or Azure Event Hub. Experience creating producers, consumers and topics and integrating them into downstream processing. Deep understanding of relational databases and CDC. Proficiency in SQL Server, Oracle or other RDBMSs; experience capturing change events using Debezium or native CDC tools and transforming them for downstream consumption. Proficiency in programming languages such as Python, Scala or Java and solid knowledge of...

Greetings from Tata Consulting Services TCS is Hiring for Data engineer Experience: 7-10 Years Location: PAN India Notice Period: 0 To 60 Days Please find the JD below Required Technical Skill - Airflow, Python, DBT, SQL

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Job Description Snowflake Data Engineer Objectives and Purpose The Senior Data Engineer ingests, builds, and supports large-scale data architectures that serve multiple downstream systems and business users. This individual supports the Data Engineer Leads and partners with Visualization on data quality and troubleshooting needs. The Senior D...

Senior Data Engineer Bangalore, India About the Role: Seeking a Senior Data Engineer with 4.5+ years of hands-on experience in designing and building scalable data pipelines on GCP using BigQuery, Airflow/Composer, Python, and SQL . The role demands strong analytical and programming skills, CI/CD (CircleCI), Terraform expertise, and a solid grasp of ETL/ELT frameworks and data governance. Key Responsibilities: Design, develop, and maintain robust ETL/ELT pipelines on GCP. Optimize and manage BigQuery datasets for performance and scalability. Automate data workflows using Terraform, CircleCI, and Airflow. Ensure data quality, security, and compliance across systems. Collaborate with cross-fun...

This job is with Standard Chartered Bank, an inclusive employer and a member of myGwork the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Job Summary Redwood RMJ Lead Key Responsibilities Processes Comply with Bank Group Policy and always adhere to Group Code of Conduct Lead/Guide/mentor team to deliver quality product by adhering to best SDLC practices People & Talent Must have experience in leading the Batch Orchestration Design, Development and end to end delivery. Strong Analytical and strategic mindset coupled with a thorough understanding of analytics in Banking Industry Hands-on experience with Redwood BPA/RMJ/Airflow/ControlM...

About Certify: At CertifyOS, we're building the infrastructure that powers the next generation of provider data products, making healthcare more efficient, accessible, and innovative. Our platform is the ultimate source of truth for provider data, offering unparalleled ease and trust while making data easily accessible and actionable for the entire healthcare ecosystem. What sets us apart? Our cutting-edge, API-first, UI-agnostic, end-to-end provider network management platform automates licensing, enrollment, credentialing, and network monitoring like never before. With direct integrations into hundreds of primary sources, we have an unbeatable advantage in enhancing visibility into the ent...

We are seeking a skilled and detail-oriented Data Engineer (Apache Flume) to join our team. The ideal candidate will have 4+ years of hands-on experience in development, with a strong grasp of Apache Flume, data modelling, and migration. You will work closely with cross-functional teams to deliver data-driven solutions that empower strategic decision-making across the organization. To lead the migration of existing logs from SQL Server databases to an Apache Flume application and establish a robust pipeline for daily log ingestion. The ideal candidate will have hands-on experience in data engineering, log management, and automation of ETL processes. Key Responsibilities Log Migration: Design...

We are seeking a highly skilled and experienced AWS Data Engineer to join our dynamic team. The ideal candidate will have a strong background in cloud-based data engineering with at least 4 years of hands-on experience in AWS . You will be responsible for designing, building, and maintaining scalable data pipelines and solutions using modern data engineering tools and frameworks. Key Responsibilities: Design and implement robust, scalable, and high-performance data pipelines on AWS. Develop and maintain ETL workflows using Airflow . Write efficient and optimized code in Python for data processing and transformation. Work with SQL to query, manipulate, and analyze large datasets. Collaborate ...

Who We Are For 20 years, we have been working with organizations large and small to help solve business challenges through technology. We bring a unique combination of engineering and strategy to Make Data Work for organizations. Our clients range from the travel and leisure industry to publishing, retail and banking. The common thread between our clients is their commitment to making data work as seen through their investment in those efforts. In our quest to solve data challenges for our clients, we work with large enterprise, cloud-based and marketing technology suites. We have a deep understanding of these solutions so we can help our clients make the most of their investment in an effic...

Job Description: This position will be responsible for design, build and maintenance of data pipelines running on Airflow, Spark on the AWS Cloud platform. Roles and Responsibility: Build and maintain all facets of Data Pipelines for Data Engineering team. Build the pipelines required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, Python and Airflow. Work with internal and external stakeholders to assist with data-related technical issues and data quality issues. Engage in proof of concepts, technical demos and interaction with customers and other technical teams Participate in the agile ceremonies Ability to solve complex data-dr...

Key Responsibilities Develop and maintain Apache Airflow DAGs for BFSI data workflows. Create custom operators, hooks, and sensors for integration with banking APIs, payment systems, and regulatory platforms. Ensure data security, compliance (PCI DSS, GDPR), and auditability in all workflows. Collaborate with QA and business teams to validate data integrity and compliance. Optimize workflows for high availability and fault tolerance in BFSI environments. Implement CI/CD pipelines for Airflow DAG deployments using Git/Jenkins. Monitor and troubleshoot Airflow jobs, ensuring timely execution of critical processes. Required Skills Strong proficiency in Python and Airflow DAG development. Hands-...

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. As a Data Engineer III at JPMorgan Chase within the Wholesale Credit Risk team, you will play a key role in transforming our legacy and fragmented data landscape, supporting advanced analytics and data-promoten decision-making across the organization. In this role, you will be an integral part of complex data engineering projects, modernize large-scale Oracle databases, on-premises big data platforms, and diverse data sources. You will leverage your technical expertise to design and implement robust data models and pipelines, optimize data sourcing strategies, and architect...

Role & responsibilities: Design, develop, and maintain data pipelines and ETL processes using Snowflake, SQL, and cloud-native tools. Create and optimize data models , ensuring scalability, performance, and data quality. Implement and manage Snowflake databases, schemas, tables, and views . Integrate data from diverse sources (e.g., AWS S3, Azure, GCP, on-prem databases, APIs). Optimize Snowflake performance using clustering, caching, and query tuning. Manage security, roles, and access controls in Snowflake. Collaborate with data analysts, data engineers, and business stakeholders to understand requirements and translate them into technical solutions. Automate and schedule data workflows us...

Experience : 6.00 + years Salary : Confidential (based on experience) Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full time Permanent Position (*Note: This is a requirement for one of Uplers client - Forbes Advisor) What do you need for this opportunity Must have skills required: Google Cloud, DPT, marketing data Forbes Advisor is Looking for: Company Description Forbes Advisor is a new initiative for consumers under the Forbes Marketplace umbrella that provides journalist- and expert-written insights, news and reviews on all things personal finance, health, business, and everyday life decisions. We do this by providing consumers with the knowledge and r...

Job Description: This position will be responsible for design, build and maintenance of data pipelines running on Airflow, Spark on the AWS Cloud platform. Roles and Responsibility: Build and maintain all facets of Data Pipelines for Data Engineering team. Build the pipelines required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, Python and Airflow. Work with internal and external stakeholders to assist with data-related technical issues and data quality issues. Engage in proof of concepts, technical demos and interaction with customers and other technical teams Participate in the agile ceremonies Ability to solve complex data-dr...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role? There are hundreds of opportunities to make your mark on technology and life at American Express. Here’s just some of wha...

Job ID: Dat-ETP-Pun-1168 Location: Pune,Bangalore,Other Company Overview Bridgnext is a Global consulting company that provides technology-empowered business solutions for world-class organizations. Our Global Workforce of over 800 consultants provide best in class services to our clients to realize their digital transformation journey. Our clients span the emerging, mid-market and enterprise space. With multiple offices worldwide, we are uniquely positioned to deliver digital solutions to our clients leveraging Microsoft, Java and Open Source with a focus on Mobility, Cloud, Data Engineering and Intelligent Automation. Emtec’s singular mission is to create “Clients for Life” – long-term rel...

You will be responsible for leading the Redwood RMJ team to deliver quality products by adhering to best SDLC practices. Your role will involve mentoring the team and ensuring compliance with Bank Group Policy and Group Code of Conduct. Additionally, you must have experience in leading Batch Orchestration design, development, and end-to-end delivery. Strong analytical and strategic mindset along with hands-on experience in Redwood BPA, RMJ, Airflow, ControlM Development, unix scripting, and database concepts are required. You should also have experience in design, development, and deployment in the DevOps environment, as well as familiarity with software like SharePoint, Confluence, ADO, and...

Job ID: Pyt-ETP-Pun-1125 Location: Pune Company Overview Bridgenext is a Global consulting company that provides technology-empowered business solutions for world-class organizations. Our Global Workforce of over 800 consultants provide best in class services to our clients to realize their digital transformation journey. Our clients span the emerging, mid-market and enterprise space. With multiple offices worldwide, we are uniquely positioned to deliver digital solutions to our clients leveraging Microsoft, Java and Open Source with a focus on Mobility, Cloud, Data Engineering and Intelligent Automation. Emtec’s singular mission is to create “Clients for Life” – long-term relationships that...

Job Title: Senior Data Engineer Location: Client Office (4 days/week), Bangalore Job Type: Full-Time About the Role: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have extensive expertise in AWS services, data engineering concepts, and programming/scripting languages. This role requires a deep understanding of cloud architecture, data processing frameworks, and data warehousing solutions. Key Responsibilities: Cloud and AWS Expertise: Utilize in-depth knowledge of AWS services such as EC2, S3, RDS, DynamoDB, Redshift, Glue, Lambda, Step Functions, Kinesis, Iceberg, EMR, and Athena. Design and implement cloud architectu...

Lead design and execution of Dataproc Databricks PySpark migration roadmap. Define modernization strategy , including data ingestion, transformation, orchestration, and governance. Architect scalable Delta Lake and Unity Catalog -based solutions. Manage and guide teams on code conversion, dependency mapping, and data validation. Collaborate with platform, infra, and DevOps teams to optimize compute costs and performance. Own the automation & GenAI acceleration layer , integrating code parsers, lineage tools, and validation utilities. Conduct performance benchmarking, cost optimization, and platform tuning (Photon, Auto-scaling, Delta Caching). Mentor senior and mid-level developers, ensuring...

As a Support Engineer, you ll be at the forefront of innovation, driving integrations with external exchanges, including social platforms and DSPs. Your work will directly impact our ability to enhance existing features and deliver exceptional solutions to our clients. We are looking for a Support Engineer to join the team. The ideal candidate will have a strong foundation in Microsoft Dynamics 365, SQL, an understanding of client-architecture, and the ability to work in a client-centric role. This role requires excellent communication skills, flexibility to support clients in US EST time zones, and a desire to continuously learn and grow. Technical Skills: Microsoft Dynamics 365, SQL, Pytho...

As a Support Engineer, you ll be at the forefront of innovation, driving integrations with external exchanges, including social platforms and DSPs. Your work will directly impact our ability to enhance existing features and deliver exceptional solutions to our clients. We are looking for a Support Engineer to join the team. The ideal candidate will have a strong foundation in Microsoft Dynamics 365, SQL, an understanding of client-architecture, and the ability to work in a client-centric role. This role requires excellent communication skills, flexibility to support clients in US EST time zones, and a desire to continuously learn and grow. Technical Skills: Microsoft Dynamics 365, SQL, Pytho...