15348 Airflow Jobs - Page 38

Experience: 4 – 8 years Must-Have: Design and develop features in the existing Data Lake/ Data Warehouse. Connect structured, semi-structured and unstructured data from sources like databases, flat files and streaming platforms. Normalise, enrich and cleanse raw data to make it usable for analytics, machine learning and reporting. Building streaming and batch pipelines Tune pipelines, queries and storage systems for speed, reliability and cost-efficiency Familiarity with data orchestration Nice to Have: AWS Cloud Platform, managed Data services such as Kinesis, S3, EMR, AWS Glue. Hands-on experience in Data modelling is required. Hands-on experience in Lakehouse / Iceberg / Parquet required....

About ProcDNA ProcDNA is a global consulting firm. We fuse design thinking with cutting-edge technology to create game-changing Commercial Analytics and Technology solutions for our clients. We're a passionate team of 275+ across 6 offices, all growing and learning together since our launch during the pandemic. Here, you won't be stuck in a cubicle you'll be out in the open water, shaping the future with brilliant minds. At ProcDNA, innovation isn't just encouraged; it's ingrained in our DNA. Ready to join our epic growth journey. What are we looking for? As a Data Scientist- I, you will play a key role in delivering end-to-end data science solutions across pharmaceutical and healthcare enga...

Important Note (Please Read Before Applying) 🚫 Do NOT apply if: • You have less than 10 years of total IT experience • You lack GCP-based data engineering project exposure • You do not have strong Python programming expertise • You are not comfortable with hybrid / client-interfacing roles • Your notice period exceeds 30 days ✅ Apply ONLY if you meet ALL the mandatory skills and experience criteria. Random or unrelated applications will not be processed. Job Title: Lead Data Engineer – Python & GCP Location: Hyderabad Experience: 10+ Years (STRICTLY) Employment Type: Permanent / Full-Time Notice Period: Immediate Joiners only About the Company: Our client is a trusted global innovator of IT ...

Who We Are Escalent is an award-winning data analytics and advisory firm that helps clients understand human and market behaviors to navigate disruption. As catalysts of progress for more than 40 years, our strategies guide the world’s leading brands. We accelerate growth by creating a seamless flow between primary, secondary, syndicated, and internal business data, providing consulting and advisory services from insights through implementation. Based on a profound understanding of what drives human beings and markets, we identify actions that build brands, enhance customer experiences, inspire product innovation and boost business productivity. We listen, learn, question, discover, innovate...

Description Position Overview : We are looking for a Business Analyst with a focus on data observability and integration projects. The candidate will bridge business and technical teams, gather and analyze requirements, and ensure high-quality data pipelines, integrations, and monitoring solutions. Key Responsibilities Collaborate with stakeholders to gather, analyze, and document business and data requirements. Translate requirements into functional and technical user stories for the data engineering team. Work on data observability use cases data quality, lineage, monitoring, and alerting frameworks. Support data integration initiatives involving APIs, ETL pipelines, and real-time streamin...

Description As the member Data Science, this is your opportunity to be part of a next-gen data science team. As a Junior Data Scientist, you'll work across business analytics, data engineering, and advanced modelling projects. You will gain hands-on experience in building clean data pipelines, generating user-level insights, and contributing to models that decode complex consumer behaviour. If you're eager to grow at the intersection of data engineering, analytics, and AI, this role is your launchpad. You will influence strategic decisions, pioneer ground breaking technologies, and deliver insights that empower businesses across industries. If you are a visionary who thrives on solving compl...

Role Overview: You will be joining the Advertising Optimization & Automation Science team at Wayfair as a Senior Machine Learning Scientist based in Bangalore. In this role, you will play a crucial part in building intelligent, ML-powered systems that drive personalized recommendations and campaign automation within Wayfair's advertising platform. Your work will involve collaborating with other scientists, as well as members of internal Product and Engineering teams, to define and deliver capabilities that enhance advertiser outcomes and unlock substantial commercial value. Key Responsibilities: - Design and build intelligent budget, tROAS, and SKU recommendations, as well as simulation-driv...

As a Machine Learning Operation Engineer at Onclusive, you will be responsible for deploying, scaling, and optimizing backend algorithms, data ingestion pipelines, machine learning services, and data platforms to support analysis on vast amounts of text and analytics data. Your technical expertise will be utilized to solve challenging marketing problems by applying Big Data analytics to Onclusive's billions of online content data points. **Key Responsibilities:** - Design and build scalable machine learning services and data platforms. - Utilize benchmarks, metrics, and monitoring to measure and improve services. - Manage systems processing tens of millions of jobs per day. - Research, desig...

Data Engineer With GCP GCP background is a must (especially BigQuery) Experience with airflow or similar orchestration tools is preferred. Knowledge of data warehousing Experience with migration projects is a plus Responsibilities: Implementing data pipelines using Python, SQL, BigQuery, airflow etc Designing and building data marts Building various business critical reports Working on various data governance areas Collaborating with various business stakeholders to work on new initiatives . Skills Java or Python or Big Query mainly to help onboard data into google The data currently is in other databases

Career Area Technology, Digital and Data Job Description Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it. Build the Digital Backbone of Modern Manufacturing. We’re assembling a dynamic team to develop and scale our Manufacturing & Supply Digital Platform—a next-g...

Job Description About KPMG in India KPMG entities in India are professional services firm(s). These Indian member firms are affiliated with KPMG International Limited. KPMG was established in India in August 1993. Our professionals leverage the global network of firms, and are conversant with local laws, regulations, markets and competition. KPMG has offices across India in Ahmedabad, Bengaluru, Chandigarh, Chennai, Gurugram, Jaipur, Hyderabad, Jaipur, Kochi, Kolkata, Mumbai, Noida, Pune, Vadodara and Vijayawada. KPMG entities in India offer services to national and international clients in India across sectors. We strive to provide rapid, performance-based, industry-focused and technology-e...

Company Description Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. How will you make an impact in this role The Global Accounts Receivables team is part of the Enterprise Platforms. The team plays a key role in the development of ...

About the Team! DE team at HYD collaborates with other engineering/product/analyst teams across tech sites to collectively accomplish OKRs to take Uber forward. We constantly enrich our data layer to effectively deal with the next generation of products which are a result of Uber's big bold bets. We design and build data pipelines to schedule & orchestrate a variety of tasks such as extract, cleanse, transform, enrich & load data as per the business needs. We serve the data insights at depth to various teams at Uber-like business analysts, data analysts, data science and business stakeholders to take strategic decisions, train DS models, perform health checks etc in order to grow and support...

About The Role As an Engineering Manager on the FinTech Data & ML Systems team , you will: Lead a high-performing team of data engineers and platform specialists in designing, implementing, and scaling data and ML solutions that power analytics, decision-making, and automation across FinTech. Drive the architecture and delivery of robust data pipelines, feature stores, and data platforms that enable machine learning and advanced analytics use cases. Collaborate closely with product managers, data scientists, and ML engineers to define and deliver reliable data and model workflows that support critical FinTech applications. Provide technical leadership in data architecture, ETL design, model ...

Company Description Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989...

Company Description Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989...

Company Description Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989...

We are looking for a hands-on, senior Snowflake expert who has both administrative and development experience in managing, securing, and optimizing large-scale Snowflake instances. The ideal candidate will be responsible for overseeing our Snowflake SaaS instance (purchased through AWS), integrating with tools such as Prefect, dbt, and Python-based data pipelines, and ensuring operational excellence, cost efficiency, and compliance. Key Responsibilities: Snowflake Administration Manage and maintain the Snowflake SaaS environment, including user roles, security, data sharing, and integrations. Monitor system performance and proactively tune configurations for optimal performance and cost effi...

Company Description Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989...

As a Software Developer you will work in a constantly evolving environment, due to technological advances and the strategic direction of the organization you work for. You will create, maintain, audit, and improve systems to meet particular needs, often as advised by a systems analyst or architect, testing both hard and software systems to diagnose and resolve system faults. The role also covers writing diagnostic programs and designing and writing code for operating systems and software to ensure efficiency. When required, you will make recommendations for future developments Benefits of Joining Us Challenging Projects : Work on cutting-edge projects and solve complex technical problems. Ca...

Greetings from TCS! Skillset : Apache Spark and Hadoop Location: Bangalore Experience Range: 4 - 10 years Must-have skills Design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc. Utilize technologies like Hadoop, Spark, Kafka, and NoSQL databases. Develop data ingestion, transformation, and aggregation strategies. Design and implement data warehouses and data marts using Presto, Snowflake, or similar technologies. Create efficient data models and schemas for optimal query performance. Great to have an understanding of graph query (GQL), Gremlin, Cypher, etc. Know to build business system-engineering ar...

Job Title: Data Engineer About the Role Were seeking a Data Engineer with 3–4 years of experience to join our growing tech team. The ideal candidate will have hands-on experience in building and managing scalable data systems on AWS, with strong expertise in data pipelines, real-time streaming, and modern data frameworks.in building and managing scalable data systems on AWS using Spark and modern data frameworks. Key Responsibilities Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of high-volume data. Develop Spark-based ETL/ELT workflows for both batch and real-time streaming data. Core Skills: Spark, AWS (S3, Glue, Redshift, Kinesis), Kafka...

Job Description **Job Description: Principal Data Architect - Principal Architect** Company:** Happiest Minds Location:** Bangalore Experience Required:** 16 - 20 years About Happiest Minds Happiest Minds is a leading digital transformation and technology services company, empowered by the mission of enhancing the happiness of our customers, employees, and society at large. We provide cutting-edge solutions leveraging advanced technologies to drive meaningful outcomes for our clients. Position Overview We are seeking an experienced and highly skilled Principal Data Architect to join our dynamic team. The ideal candidate will possess a solid background in data architecture, data lakes, and ET...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. About LUMI: LUMI is the company’s largest Big Data Platform, purpose-built for computationally and data-intensive processing applications. Whether data is processe...