19653 Airflow Jobs - Page 13

Company Description Attributics is a cutting-edge MarTech and Agentic AI startup, revolutionizing the way brands personalize and optimize consumer experiences. Specializing in customer data platforms (CDPs), multi-channel engagement, AI-driven journey orchestration, and advanced analytics, Attributics powers growth through deep segmentation, product recommendations, and automation. As certified partners with leading platforms like Segment, Braze, and Azure, Attributics delivers secure, scalable, and innovative marketing solutions across MENA, India, and APJ regions. By combining expertise in data engineering, modern data stack implementations, and AI technologies, Attributics is at the foref...

As a data engineer - YOU GET TO WORK ON- Data modeling for OLAP, warehousing and building ETL pipelines on cloud AWS technologies like EMR, Lambda, S3, SQS, EC2, Glue, etc. Using Airflow for orchestration BASIC QUALIFICATIONS 1-3 years of data engineering experience Experience with SQL and NoSQL databases Practical Knowledge of Linux or Unix shell scripting. Experience in at least one programming language, such as Python or Javascript Experience partnering with business owners directly to understand their requirements and provide data which can help them observe patterns and spot anomalies PREFERRED QUALIFICATIONS Experience with emerging technologies in space like Kafka, ClickHouse Experien...

We are seeking a highly motivated and expert Senior Data Engineer for Framework Development who will lead the design and development of enterprise-grade testing frameworks, data quality (DQ) frameworks, and automation frameworks to strengthen the reliability and scalability of Amgens enterprise data ecosystem. The ideal candidate will architect reusable frameworks that enable efficient validation, continuous testing, data observability, and governance automation across data pipelines, analytics models, and platform components. This role requires deep expertise in test automation, data validation strategies, software engineering best practices, and modern data platforms . The candidate will w...

National Payment Corporation of India (NPCI) is hiring! Job Title: Data Engineer Division: NPCI Market Innovation Years of Experience: 3+ to 8 years Education: Bachelors/Master’s in Computer Science, Electrical Engineering, Data Science, or related technical field Employment Type: Full-time Location: Mumbai, Hyderabad & Chennai Role Type: Permanent Overview:- NPCI is looking for Data Engineers, who are well versed with the Big data technologies. The candidate will be working on-premises handling the massive data, that will be served to the data science and data modelling team. He / She is responsible for data availability in the existing data pipelines also should be able to establish and ma...

We are seeking an experienced ETL/SQL/Python Testing Engineer to design, develop, and execute test cases for data pipelines, ETL processes, and database systems. The ideal candidate will possess strong expertise in SQL querying, ETL validation, Python scripting, and test automation to ensure data accuracy, completeness, and optimal performance. Your future duties and responsibilities: Develop and execute ETL test cases to validate data extraction, transformation, and loading processes Perform ETL pipeline testing in an ADB (Azure Databricks) environment Write complex SQL queries to verify data integrity, consistency, and correctness across source and target systems Automate ETL testing workf...

Hands-on individual responsible for producing excellent quality of code, adhering to expected coding standards and industry best practices. Must have strong experience in Java, Python , Airflow exp microservices, Spring and SQL. Good knowledge of Spring, Hibernate, JMS, Caching Frameworks, Memory Management Nice to have Kubernetes , Docker. Problem- solving/ Trouble shooting skills High levels of ownership and commitment on deliverables Strong Communication Skills - Should be interact with client stakeholders to probe a technical problem or clarify requirement specifications Mandatory Competencies Programming Language - Java - Core Java (java 8+) Programming Language - Java - Spring Framewor...

We are seeking an experienced and Senior DevOps Engineer to lead the deployment of Docker container-based applications and API-driven microservices built in Python , running on Kubernetes within our Ascend Platform . This role will be instrumental in driving innovation, streamlining deployment workflows, and enhancing team efficiency through established CI/CD pipelines and industry best practices. Key Responsibilities: DevOps Leadership : Architect and manage scalable, secure, and resilient infrastructure for containerized applications and microservices. CI/CD Ownership : Maintain and evolve CI/CD pipelines to ensure rapid and reliable delivery of software. Platform Expertise : Oversee deplo...

• 2+ years of experience in Big Data and Data Engineering, building enterprise-level applications in a public cloud, preferably Google Cloud • Experience building cloud-native data pipelines using Python, Apache Airflow, Apache Spark, and Apache Beam. • Working knowledge of SQL and data warehousing concepts, including PostgreSQL and BigQuery. Optionally, knowledge of Mongo and AlloyDB is a plus. • Comfortable using Azure DevOps or similar CI/CD tools, Git. • Experience with TDD, agile software development processes, and collaborating in multi-functional agile teams. • Communicate effectively with technical teams and non-technical stakeholders. • Bachelor's degree in Computer Science or relat...

Role Description Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc with coding skills in Python PySpark and SQL. Works independently and demonstrates proficiency in at least one domain related to data with a solid understanding of SCD concepts and data warehousing principles. Outcomes Collaborate closely with data analysts data scientists and other stakeholders to ensure data accessibility quality and security across various data sources.rnDesign develop and maintai...

Position Title D&T Machine Learning Engineer I Function/Group Digital and Technology Country India Location: Mumbai Role Reports To D&T Manager – ML Engineering Geographical Scope of the Role - Location: Global Percentage Of Travel Required 1% International Remote/Hybrid/in- Office Hybrid On-call Required Yes Shift Regular About General Mills We make food the world loves: 100 brands. In 100 countries. Across six continents. With iconic brands like Cheerios, Pillsbury, Betty Crocker, Nature Valley, and Häagen-Dazs, we’ve been serving up food the world loves for 155 years (and counting). Each of our brands has a unique story to tell. How We Make Our Food Is As Important As The Food We Make. Ou...

About The Opportunity Operating in the IT services & analytics sector, this role sits inside a data engineering practice that builds large-scale batch and streaming data platforms for enterprise clients. We deliver high-throughput, low-latency data processing solutions using Hadoop ecosystems and modern ETL practices to support analytics, reporting and ML use cases. Position: Big Data Engineer (Hadoop Developer) | Location: India | Workplace: On-site Role & Responsibilities Design, develop and maintain end-to-end Hadoop-based data pipelines for batch and streaming ingestion, transformation, and delivery to downstream consumers. Implement and optimise Spark and MapReduce jobs for high-volume ...

Grid Dynamics wants to build a centralized, observable and secure platform for their ML, Computer Vision, LLM and SLM models. Grid Dynamics wants to onboard a vast number of AI agents, able to cover multiple required skills, ensuring a certain level of control and security in regards to their usage and availability. The observable platform must be vendor-agnostic, easy to extend to multiple type of AI applications and flexible in terms of technologies, frameworks and data types. This project is focused on establishing a centralized LLMOps capability where every ML, CV, AI-enabled application is monitored, observed, secured and provides logs of every activity. The solution consists of key bui...

About The Opportunity Operating in the IT services & analytics sector, this role sits inside a data engineering practice that builds large-scale batch and streaming data platforms for enterprise clients. We deliver high-throughput, low-latency data processing solutions using Hadoop ecosystems and modern ETL practices to support analytics, reporting and ML use cases. Position: Big Data Engineer (Hadoop Developer) | Location: India | Workplace: On-site Role & Responsibilities Design, develop and maintain end-to-end Hadoop-based data pipelines for batch and streaming ingestion, transformation, and delivery to downstream consumers. Implement and optimise Spark and MapReduce jobs for high-volume ...

About The Opportunity Operating in the IT services & analytics sector, this role sits inside a data engineering practice that builds large-scale batch and streaming data platforms for enterprise clients. We deliver high-throughput, low-latency data processing solutions using Hadoop ecosystems and modern ETL practices to support analytics, reporting and ML use cases. Position: Big Data Engineer (Hadoop Developer) | Location: India | Workplace: On-site Role & Responsibilities Design, develop and maintain end-to-end Hadoop-based data pipelines for batch and streaming ingestion, transformation, and delivery to downstream consumers. Implement and optimise Spark and MapReduce jobs for high-volume ...

🚀 We’re Hiring: Senior Databricks Developer 📍 Location : Baner, Pune (Onsite, 5 days a week) 🏢 Work Mode : Full-time | Office-based ⏳ Availability : Immediate Joiners / Notice Period ≤ 30 Days ⚠️ Note : local Pune candidates are preferred. 🧭 L2 interviews will be conducted in person — apply only if you are open to face-to-face interviews. 🌟 About Newscape Consulting- Newscape is a fast-growing digital services company specializing in healthcare technology and data engineering. We help healthcare organizations modernize legacy systems, integrate cloud-native platforms, and drive digital transformation through data. If you’re passionate about solving complex data challenges and contributing to...

About The Opportunity Operating in the IT services & analytics sector, this role sits inside a data engineering practice that builds large-scale batch and streaming data platforms for enterprise clients. We deliver high-throughput, low-latency data processing solutions using Hadoop ecosystems and modern ETL practices to support analytics, reporting and ML use cases. Position: Big Data Engineer (Hadoop Developer) | Location: India | Workplace: On-site Role & Responsibilities Design, develop and maintain end-to-end Hadoop-based data pipelines for batch and streaming ingestion, transformation, and delivery to downstream consumers. Implement and optimise Spark and MapReduce jobs for high-volume ...

About The Opportunity Operating in the IT services & analytics sector, this role sits inside a data engineering practice that builds large-scale batch and streaming data platforms for enterprise clients. We deliver high-throughput, low-latency data processing solutions using Hadoop ecosystems and modern ETL practices to support analytics, reporting and ML use cases. Position: Big Data Engineer (Hadoop Developer) | Location: India | Workplace: On-site Role & Responsibilities Design, develop and maintain end-to-end Hadoop-based data pipelines for batch and streaming ingestion, transformation, and delivery to downstream consumers. Implement and optimise Spark and MapReduce jobs for high-volume ...

About The Opportunity Operating in the IT services & analytics sector, this role sits inside a data engineering practice that builds large-scale batch and streaming data platforms for enterprise clients. We deliver high-throughput, low-latency data processing solutions using Hadoop ecosystems and modern ETL practices to support analytics, reporting and ML use cases. Position: Big Data Engineer (Hadoop Developer) | Location: India | Workplace: On-site Role & Responsibilities Design, develop and maintain end-to-end Hadoop-based data pipelines for batch and streaming ingestion, transformation, and delivery to downstream consumers. Implement and optimise Spark and MapReduce jobs for high-volume ...

Job Title: Python Developer Location: Bengaluru, Hyderabad, Chennai Work Mode: Hybrid Experience: 5+ years Key Skills 5+ years of hands-on Python experience with strong core Python and design patterns. Expertise in Django , and familiarity with FastAPI , Flask , and Pydantic . Experience with data analysis tools: NumPy, Pandas, Polars, Dask . Knowledge of data pipeline frameworks such as Airflow, Beam, Luigi (added advantage). Strong SQL skills and understanding of RDBMS concepts. Good understanding of Agile development practices. Professional communication, client-focused mindset, and eagerness to learn. Additional Responsibilities Mentor and guide team members , contributing to their techn...

Eligibility & Requirements Common Requirements 1. Strong programming skills in Python (Java/R for some roles). 2. Expertise in SQL and relational databases (Oracle, PostgreSQL, SQL Server). 3. Bachelor’s/Master’s degree in Computer Science, IT, Statistics, or related field. 4. Excellent communication and problem-solving skills. Eligibility & Requirements - Data Scientist 1. 5–18 years of experience in Data Science. 2. Strong knowledge of ML algorithms (Regression, Random Forest, SVM, Neural Networks). 3. Exposure to Deep Learning, NLP, and Computer Vision. Eligibility & Requirements - Azure Data Engineer 1. 5–18 years of experience in Data Engineering. 2. Hands-on with Azure ecosystem (ADF, ...

About MBRDI: Mercedes-Benz Research and Development India (MBRDI), headquartered in Bengaluru with a satellite office in Pune, is the largest R&D center for Mercedes-Benz Group AG outside of Germany. Our mission is to drive innovation and excellence in automotive engineering, digitalization, and sustainable mobility solutions, shaping the future of mobility. About the Role We are looking for a Data Scientist who is highly skilled in Python, Large Language Models (LLMs), Orchestrator Frameworks, and Multi-Agent Systems to build scalable, enterprise-grade AI solutions. The ideal candidate is hands-on, can convert real-world business problems into AI-driven workflows, and is comfortable working...

Locations - Indore / Noida / Bengaluru / Pune /Gurgaon Job Description 2–5 years of hands-on experience as a Data Engineer Strong experience with Databricks and Apache Spark Proficiency in Python for data engineering and ETL development Experience building and managing ETL pipelines Experience with cloud platforms ( AWS / Azure / GCP ) – at least one Familiarity with Delta Lake and optimized data storage formats Understanding of Generative AI fundamentals (LLMs, embeddings, RAG concepts) Good to Have Hands-on experience with GenAI frameworks (Lang Chain, Lang graph, etc.) Experience working with vector databases (PGVector, Pinecone, Chroma, etc.) Knowledge of data orchestration tools (Airflo...

Work Location: Hyderabad Experience: 15+ Years Notice: Immedaite- 15 Days About SIDGS: SIDGS is a premium global systems integrator and global implementation partner of Google corporation, providing Digital Solutions & Services to Fortune 500 companies. Our Digital solutions go across following domains: User Experience, CMS, API Management, Microservices, DevOps, Cloud, Service Mesh, Artificial Intelligence, RPA domains. We create innovative solutions in Digital, API Management, Cloud and DevOps space in partnership with Google. We understand that every business has a unique set of challenges and opportunities, and we leverage our unique industry insights, honed through decades of combined e...

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. Primary Responsibilities Develop high-quality software design and architecture Prioritize and execute tasks in the ...

Skills and Experience Required: 5+ years of Python experience. Core Python expertise along with a strong understanding of design patterns. Familiarity with data analysis tools such as Numpy, Pandas, Polars, and Dask. Experience with Rest API frameworks like FastAPI, Flask, and Pydantic. Data pipeline frameworks knowledge (e.g., Airflow, Beam, Luigi) is a plus. SQL knowledge for advanced queries and understanding of RDBMS concepts is beneficial. Professionalism, a focus on client needs, and an eagerness to learn new skills. A solid understanding of Agile development practices.