10863 Airflow Jobs - Page 46

Key Responsibilities Design, develop, and optimize ETL/ELT pipelines . Manage batch and real-time (streaming) data ingestion using modern frameworks. Ensure data quality, integrity, and consistency across systems. Collaborate with data scientists and ML engineers to provide reliable datasets. Maintain data documentation, metadata, and lineage for transparency and governance. Must-Have Skills Strong proficiency in SQL/NoSQL databases . Hands-on experience with Apache Spark and Apache Kafka . Expertise in Airflow (or similar orchestration tools). Solid experience in ETL/ELT pipeline development . Good-to-Have Skills Exposure to vector databases (e.g., Milvus, Pinecone). Experience with data li...

🌟 We're Hiring: Data Platform Engineer! 🌟 We are seeking an experienced Data Platform Engineer to design, build, and maintain scalable data infrastructure using AWS cloud services. The ideal candidate will have expertise in Python, PySpark, EMR, and Apache Airflow to develop robust data pipelines and analytics solutions that drive business insights. 📍 Location: Pune, India ⏰ Work Mode: Work from anywhere 💼 Role: AWS, Python, Pyspark, EMR, Apache Airflow- Data Platform Engineer What You'll Do 🎯 Design and implement scalable data pipelines using Apache Airflow ☁️ Build and optimize AWS EMR clusters for big data processing 🐍 Develop data processing applications using Python and PySpark 📊 Create...

Who we are: Welcome to Archipelago, where we're redefining the landscape of commercial property and casualty insurance with the most powerful AI Agent that helps drive meaningful business decisions and value for their clients as easily and efficiently as possible. By equipping brokers with the Agent, Hub, and Platform we empower our customers to navigate the complexities of property and casualty insurance with confidence and control. Archipelago was founded in 2018 and serves many of the world's largest commercial P&C brokers and their clients, representing over 500 of the world's largest and most dynamic portfolios to improve their data and better represent their risks. Archipelago has achi...

Title: Senior Data Engineer / ETL Engineer Location: Pune / Hyderabad / Mohali Job Summary: As a Senior Data Engineer / ETL Engineer, you will be instrumental in designing, developing, and optimizing data processing systems that support our organization's data initiatives. Your expertise in Bigdata and Google Cloud Platform (GCP) will be essential in building robust ETL pipelines that transform raw data into actionable insights. You will collaborate with cross-functional teams to ensure that data is accurate, accessible, and valuable for decision-making. Responsibilities: Design and Develop ETL Processes: Lead the design and implementation of ETL processes using all kinds of batch/streaming ...

AIML Engineer– Global Data Analytics, Technology (Maersk) This position will be based in India – Bangalore/Pune A.P. Moller - Maersk A.P. Moller – Maersk is the global leader in container shipping services. The business operates in 130 countries and employs 80,000 staff. An integrated container logistics company, Maersk aims to connect and simplify its customers’ supply chains. Today, we have more than 180 nationalities represented in our workforce across 131 Countries and this mean, we have elevated level of responsibility to continue to build inclusive workforce that is truly representative of our customers and their customers and our vendor partners too. We are responsible for moving 20 %...

Your IT Future, Delivered. Senior Software Engineer (AI/ML Engineer) With a global team of 5600+ IT professionals, DHL IT Services connects people and keeps the global economy running by continuously innovating and creating sustainable digital solutions. We work beyond global borders and push boundaries across all dimensions of logistics. You can leave your mark shaping the technology backbone of the biggest logistics company of the world. All our offices have earned #GreatPlaceToWork certification, reflecting our commitment to exceptional employee experiences. Digitalization. Simply delivered. At DHL IT Services, we are designing, building and running IT solutions for the whole DPDHL global...

There's likely a reason you've taken the time out of your busy day to review this opportunity at PulsePoint. Maybe you're in need of a change or there's an itch you're looking to scratch. Whatever may be the reason, listen to what some of our team members are saying about working here: My manager takes the time to not only identify my next career move, but the steps that will take me there. I even have my own personal training budget that I'm encouraged to spend. Our input is valued and considered. Everyone has a voice and that goes a long way in ensuring that we're moving towards a shared goal . The Leadership team is incredibly open on their goals and how I contribute to the larger company...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Manager Job Description & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data trans...

Technical Specialist - Data Engineer Position Overview Job Title: Technical Specialist - Data Engineer Corporate Title: Associate Location: Pune, India Role Description An Engineer is responsible for designing, developing and delivering significant components of engineering solutions to accomplish business goals. Key responsibilities of this role include active participation in the design and development of new features of application enhancement, investigating re-use, ensuring that solutions are fit for purpose and maintainable, and can be integrated successfully into the overall solution and environment with clear, robust and well tested deployments. Assists more junior members of the team...

At Sogeti, we believe the best is inside every one of us. Whether you are early in your career or at the top of your game, well encourage you to fulfill your potentialto be better. Through our shared passion for technology, our entrepreneurial culture , and our focus on continuous learning, well provide everything you need to doyour best work and become the best you can be. Part of the Capgemini Group, Sogeti makes business value through technology for organizations that need to implement innovation at speed and want a localpartner with global scale. With a hands-on culture and close proximity to its clients, Sogeti implements solutions that will help organizations work faster, better, andsm...

Lead Data Scientist TBO is a global platform that aims to simplify all buying and selling travel needs of travel partners across the world. The proprietary technology platform aims to simplify the demands of the complex world of global travel by seamlessly connecting the highly distributed travel buyers and travel suppliers at scale. The TBO journey began in 2006 with a simple goal to address the evolving needs of travel buyers and suppliers, and what started off as a single product air ticketing company, has today become the leading B2A (Business to Agents) travel portal across the Americas, UK & Europe, Africa, Middle East, India, and Asia Pacific. Today, TBO's product range from air, hote...

Senior Data Scientist Location: Remote (Sri Lanka or Global flexible timezone overlap preferred) Type: Full-time Salary: $4,000 $6,000/month Company: Huex About Huex Huex helps global companies build high-performance product teams using Sri Lanka's top 3% talent , enhanced by our proprietary aiBoost framework . From healthcare and fintech to real estate and AI startups, our teams deliver scalable, innovative solutions at startup speed. We're now looking for a Senior Data Scientist to lead high-impact data science projects across our client basebuilding intelligent systems, AI-integrated features, and performance-driven models that accelerate decision-making and product outcomes. What You'll ...

We are seeking a highly skilled Senior PySpark Data Engineer to design, build, and optimize large-scale data pipelines and distributed systems. Beyond deep expertise in Apache Spark (PySpark) and automation, this role requires the ability to manage stakeholders, ensure timely delivery, and assess requirements. You will play a critical role in bridging business needs with technical execution, ensuring high-quality, scalable, and reliable data solutions. KEY RESPONSIBLITIES Technical Delivery Data Pipeline Development: Design, develop, and optimize scalable ETL/ELT pipelines using PySpark. System Integration & Automation: Build automated data ingestion and transformation frameworks integra...

Company Description Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them save time and money. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments. We invest in people and new advanced technologies to unlock the power of data. A...

You are passionate about quality and how customers experience the products you test. You have the ability to create, maintain and execute test plans in order to verify requirements. As a Quality Engineer at Equifax, you will be a catalyst in both the development and the testing of high priority initiatives. You will develop and test new products to support technology operations while maintaining exemplary standards. As a collaborative member of the team, you will deliver QA services (code quality, testing services, performance engineering, development collaboration and continuous integration). You will conduct quality control tests in order to ensure full compliance with specified standards ...

Role - Data Engineer Experience - 2+yrs Location - Bangalore Roles and Responsibilities: Create and maintain optimal data pipeline architecture Create and maintain events/streaming based architecture/design Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies. Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industr...

About Us Udacity is now an Accenture company, and exciting things are happening! 🚀 We are on a mission of forging futures in tech through radical talent transformation in digital technologies. We offer a unique and immersive online learning platform, powering corporate technical training in fields such as Artificial Intelligence, Machine Learning, Data Science, Autonomous Systems, Cloud Computing and more. Our rapidly growing global organization is revolutionizing how the enterprise market bridges the talent shortage and skills gaps during their digital transformation journey. Udacity's mission is to democratize education. We're an online learning platform offering groundbreaking education i...

Company Description Forbes Advisor is a new initiative for consumers under the Forbes Marketplace umbrella that provides journalist- and expert-written insights, news and reviews on all things personal finance. We are an experienced team of industry experts dedicated to helping readers make smart decisions and choose the right products with ease. Marketplace boasts decades of experience across dozens of geographies and teams. The team brings rich industry knowledge to Marketplace's global coverage of consumer credit, debt, health, home improvement, banking, investing, credit cards, small business, education, insurance, loans, real estate and travel. Job Description The Role We're hiring a Da...

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Generative AI Consultant for Supply Chain Business Process – Senior Consultant The opportunity We’re looking for Senior Level Consultants with expertise in Generative AI who can work on end-to-end pipelines, enabling data curation, building and fine-tuning Generative AI models, and deploying them into scalable production streams. Analytics, A...

Key Responsibilities: Design, develop, and maintain scalable ETL/ELT pipelines in Snowflake to support data migration from legacy systems. Leverage Python for data transformation, automation, and orchestration of migration workflows. Optimize and refactor complex SQL queries to ensure efficient data processing and reporting in Snowflake. Collaborate on data modeling and schema design to align with Snowflake architecture and performance best practices. Monitor and troubleshoot data pipeline performance during and after migration phases. Work closely with data analysts, scientists, and business stakeholders to ensure accurate and timely data delivery. Implement and enforce data governance, sec...

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Generative AI Consultant for Supply Chain Business Process – Senior Consultant The opportunity We’re looking for Senior Level Consultants with expertise in Generative AI who can work on end-to-end pipelines, enabling data curation, building and fine-tuning Generative AI models, and deploying them into scalable production streams. Analytics, A...

Job Overview We are seeking a skilled and detail-oriented HVAC Engineer with experience in cleanroom HVAC systems, including ducting, mechanical piping, and sheet metal works. The ideal candidate will assist in site execution, technical coordination, and quality assurance in line with cleanroom standards for pharmaceutical, biotech, or industrial facilities. Key Responsibilities : Support end-to-end HVAC system execution, including ducting, AHU installation, chilled water piping, and insulation. Supervise and coordinate day-to-day HVAC activities at the site in line with approved drawings and technical specifications. Review and interpret HVAC layouts, shop drawings, and coordination drawing...

About Us: HighLevel is an AI powered, all-in-one white-label sales & marketing platform that empowers agencies, entrepreneurs, and businesses to elevate their digital presence and drive growth. We are proud to support a global and growing community of over 2 million businesses, comprised of agencies, consultants, and businesses of all sizes and industries. HighLevel empowers users with all the tools needed to capture, nurture, and close new leads into repeat customers. As of mid 2025, HighLevel processes over 4 billion API hits and handles more than 2.5 billion message events every day. Our platform manages over 470 terabytes of data distributed across five databases, operates with a network...

Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity. Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increase data accessibility, and fostering data-driven decision making across the organization. Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it Writes unit/integration tests, contributes to engineering wiki, and documents work. Designs and develops data integrations and data quality framework Bachelor of Engineering or equivalent degree in Computer S...

About Entrupy Entrupy is a global technology company whose mission is to protect businesses, borders and consumers from transacting in counterfeit goods. Entrupy has developed a patented technology system which utilizes a combination of AI and computer vision to instantly identify and authenticate high value physical goods. Entrupy's solutions serve business customers including leading luxury brands, retailers, e-commerce marketplaces and online resellers in over 60 countries. Entrupy is growing quickly with team members based in the US, India, Japan and Brazil. Entrupy's solutions in market: Entrupy Luxury Authentication Entrupy Sneaker Authentication Entrupy Fingerprinting As we continue t...