15348 Airflow Jobs - Page 19

Role Overview: As an AI Data Engineer, your main responsibility will be to build scalable, high-quality data pipelines that support advanced analytics and AI/ML models. You will play a crucial role in transforming raw data into reliable datasets that are ready for AI applications. If you find satisfaction in working with data and turning it into valuable insights, this role is tailored for you. Key Responsibilities: - Design, build, and maintain end-to-end data pipelines specifically for AI/ML workloads - Develop high-quality datasets suitable for modeling by transforming, cleaning, and enriching raw data - Manage feature stores and assist ML engineers with feature engineering - Collaborate ...

Role Overview: In this role as a Data COE Data Engineer at Mondelz International, you will have the opportunity to design and build scalable, secure, and cost-effective cloud-based data solutions. You will be responsible for developing and maintaining data pipelines to extract, transform, and load data into data warehouses or data lakes, ensuring data quality and validation processes to maintain data accuracy and integrity. Collaboration with data teams, product owners, and other stakeholders to stay updated with the latest cloud technologies and best practices is essential. Key Responsibilities: - Design and Build: Develop and implement scalable, secure, and cost-effective cloud-based data ...

Role Overview: You will be responsible for working with machine learning and natural language processing in document-heavy domains, particularly in finance, legal, and policy. Your main focus will be on data ingestion and preprocessing, NLP modeling, knowledge graph engineering, information retrieval, annotation and validation, MLOps, API integration, and more. Key Responsibilities: - Data Ingestion and Preprocessing: - Build and maintain data pipelines to ingest unstructured data from PDFs, gazettes, HTML circulars, etc. - Process data extraction, parsing, and normalization. - NLP & LLM Modeling: - Fine-tune or prompt-tune LLMs for summarization, classification, and change detection in regu...

You have an exciting opportunity for a role where you can utilize your 5+ years of experience in GCP - Data Side, Storage Classes, Dataflow, BigQuery, Pyspark/Python, and Airflow. Your expertise in these areas will be crucial as you take on the following key responsibilities: - Work on GCP platforms focusing on Data Side and Storage Classes - Utilize Dataflow for efficient data processing - Manage and query large datasets using BigQuery - Develop data pipelines using Pyspark and Python - Implement and schedule workflows with Airflow To excel in this role, you will need the following qualifications: - 5+ years of experience in working with GCP technologies - Proficiency in Dataflow, BigQuery,...

You will be involved in providing advanced analytical capabilities to support data science initiatives. This role includes experience in various areas such as predictive modeling, time series and demand forecasting, optimization, linear and non-linear curve fitting, price elasticity, feature engineering, model development and management, production deployment of models, scalability of systems, and measurement in the realm of price optimization, dynamic pricing, and promotion analytics. As a Senior MLE, you will solve complex problems that require deploying models at scale. **Roles And Responsibilities:** - Apply advanced data science, machine learning (ML), and deep learning (DL) algorithms ...

We are looking for a hands-on, senior Snowflake expert who has both administrative and development experience in managing, securing, and optimizing large-scale Snowflake instances. The ideal candidate will be responsible for overseeing our Snowflake SaaS instance (purchased through AWS), integrating with tools such as Prefect, dbt, and Python-based data pipelines, and ensuring operational excellence, cost efficiency, and compliance. Key Responsibilities: Snowflake Administration Manage and maintain the Snowflake SaaS environment, including user roles, security, data sharing, and integrations. Monitor system performance and proactively tune configurations for optimal performance and cost effi...

Job Description: Key Roles And Responsibilities Design, develop, and maintain robust data pipelines and AI integration solutions using Python on Azure Databricks. Integrate Generative AI and Agentic AI models into existing and new workflows to drive business innovation and automation. Collaborate with data scientists, AI researchers, software engineers, and product teams to deliver scalable and efficient AI-powered solutions. Orchestrate data movement and transformation across Azure-native services including Azure Databricks, Azure Storage (Blob, Data Lake), and Snowflake, ensuring data quality, security, and compliance. Integrate enterprise data using Palantir Foundry and leverage Azure ser...

Job Description: Key Responsibilities Design, develop, and maintain robust data pipelines and AI integration solutions using Python on Azure Databricks. Integrate Generative AI and Agentic AI models into existing and new workflows to drive business innovation and automation. Collaborate with data scientists, AI researchers, software engineers, and product teams to deliver scalable and efficient AI-powered solutions. Orchestrate data movement and transformation across Azure-native services including Azure Databricks, Azure Storage (Blob, Data Lake), and Snowflake, ensuring data quality, security, and compliance. Integrate enterprise data using Palantir Foundry and leverage Azure services for ...

Data Platform - Design and build an in house data warehouse for near real time data sharing multiple products, including partnerships, multi-tenancy and RBAC Preferably someone with experience using these technologies like: Flink, Airflow, Hudi, Delta

12+ years of experience in data engineering, data architecture, or related fields. 5+ years in a leadership or practice lead role managing teams and strategic initiatives. Strong expertise in cloud data platforms (Azure, AWS, or GCP). Proven experience with data lake , data warehouse , and ETL/ELT architectures. Hands-on experience with tools such as Databricks, Snowflake, Apache Spark, Kafka, Airflow , etc. Knowledge of data governance , security , and metadata management frameworks. Familiarity with DevOps , CI/CD , and DataOps practices. Excellent communication, stakeholder management, and leadership skills.

Must have 3-5 years of hands-on experience on Data Engineering, Cloud, DevOps and AI/Model Engineering across the following technology stack o Advanced SQL o PySpark o Cloud services o Airflow or an equivalent cloud native orchestration tool o Kubernetes or an equivalent cloud native container service o Microservices o DevOps o Mlflow an equivalent cloud native model deployment and monitoring service o Basics of Generative AI --Large Language Models --Prompt Engineering --LangChain/LangGrapho Basics of Responsible AI Knowledge of design principles and fundamentals, Understanding of performance optimization/automations, Basic understanding of project domain, Good understanding of AI/ML deploy...

Medidata Solutions is seeking a Senior DataOps Engineer to help scale and operate our next-generation clinical data platform. You will work closely with Data Engineering, Data Services and AI/ML teams to automate deployments, enhance observability, and ensure reliable operations for critical pipelines and services powering CTMS, EDC, and eTMF integrations. What You will Do Automate deployment and monitoring of Snowflake, Airflow, and S3/Iceberg pipelines across multiple environments. Build and maintain CI/CD workflows using GitHub Actions and infrastructure as code with Terraform. Deploy, operate, and scale Python and Java applications on AWS services, including Lambda, ECS, EKS, and Step Fu...

Expedia Group brands power global travel for everyone, everywhere. We design cutting-edge tech to make travel smoother and more memorable, and we create groundbreaking solutions for our partners. Our diverse, vibrant, and welcoming community is essential in driving our success. Why Join Us To shape the future of travel, people must come first. Guided by our Values and Leadership Agreements, we foster an open culture where everyone belongs, differences are celebrated and know that when one of us wins, we all win. We provide a full benefits package, including exciting travel perks, generous time-off, parental leave, a flexible work model (with some pretty cool offices), and career development ...

Data Engineering Lead Analyst Position Overview Evernorth Core Platform Engineering team is looking for a Data Engineer to design, develop, and implement robust data pipelines for Packaged Business Capabilities (PBCs). You'll work closely with data scientists, analysts, and other engineers to ensure high data quality, performance, and availability. The ideal candidate brings hands-on experience working with big data tools and platforms such as Apache Spark and Databricks, has a strong command of Python, and is comfortable managing ETL workflows, DAGs, and orchestration using tools like Jenkins or Airflow. Experience in cloud-native environmentsparticularly AWS and AWS Glue is essential. Abou...

About Position: We're looking for an AWS Data Platform Engineer to help automate and scale our cloud-based analytics environment. You'll work with our BI and Data Engineering teams to build secure, automated, and reusable AWS infrastructure that supports data pipelines, Redshift environments, and business intelligence integrations. Role: AWS Data Platform Engineer Location: All Persistent Location Experience: 9+years Job Type: Full Time Employment What You'll Do: Design and maintain CI/CD pipelines for data platform and analytics workloads. Build and manage containerized applications using Docker, ECS, and ECR. Develop Infrastructure as Code (Terraform or AWS CDK) for Redshift, Lambda, and o...

Some careers shine brighter than others. If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organizations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realize their ambitions. We are seeking for the role of Senior Software Engineer. In This Role Y...

Position Summary We are looking for a highly skilled and experienced Data Engineering Manager to lead our data engineering team. The ideal candidate will possess a strong technical background, strong project management abilities, and excellent client handling/stakeholder management skills. This role requires a strategic thinker who can drive the design, development and implementation of data solutions that meet our clients needs while ensuring the highest standards of quality and efficiency. Job Responsibilities Technology Leadership Lead guide the team independently or with little support to design, implement deliver complex cloud-based data engineering / data warehousing project assignment...

As an AI/ML Engineer, you will be responsible for leveraging your expertise in AI/ML to develop cutting-edge solutions. Here is a breakdown of what the role entails: **Key Responsibilities:** - Utilize your 6+ years of experience in AI/ML to develop innovative solutions - Demonstrate proficiency in Python and various ML libraries including scikit-learn, XGBoost, pandas, NumPy, matplotlib, and seaborn - Apply a strong understanding of machine learning algorithms and deep learning architectures such as CNNs, RNNs, and Transformers - Hands-on experience with TensorFlow, PyTorch, or Keras - Engage in data preprocessing, feature selection, EDA, and model interpretability - Comfortably develop API...

As a Lead Analytics Engineer, your role will involve taking ownership of the analytics data Pipeline and playing a pivotal role in designing and scaling the modern data stack. You will be responsible for shaping data models in dbt/Snowflake, orchestrating pipelines using Airflow, and enabling high-quality, trusted data for reporting. Key Responsibilities: - Lead the development and governance of DAZNs semantic data models to support consistent, reusable reporting metrics. - Architect efficient, scalable data transformations on Snowflake using SQL/DBT and best practices in data warehousing. - Manage and enhance pipeline orchestration with Airflow, ensuring timely and reliable data delivery. -...

Position: Data Scientist Location: Pune / Hyderabad / Bangalore Experience: 4–5 Years About the Role: We’re looking for a Data Scientist with strong analytical and programming skills to develop and deploy predictive models, perform statistical analysis, and generate actionable insights from large datasets. The ideal candidate will have hands-on experience with Python, ML algorithms, visualization tools, and cloud-based AI platforms . Key Responsibilities: Build and deploy predictive models using machine learning algorithms and statistical techniques . Perform data preprocessing, feature engineering, and model evaluation. Visualize trends and insights using Power BI, Tableau, or Python librar...

As a Data Engineer Senior Consultant at EY, you will be responsible for designing, building, and optimizing data solutions to enable advanced analytics and AI-driven business transformation. Your role will involve working with modern data engineering practices and cloud platforms to deliver robust and scalable data pipelines for various business domains such as finance, supply chain, energy, and commercial operations. Key Responsibilities: - Design, develop, and deploy end-to-end data pipelines for complex business problems, supporting analytics and modernizing data infrastructure. - Design and implement data models, ETL/ELT workflows, and data integration solutions across structured and uns...

As a Technical Lead in Product Engineering at Saarthee, you will play a crucial role in leading the end-to-end technical development lifecycle of cutting-edge technology products. Your responsibilities and duties will include: - Leading the end-to-end technical development lifecycle including architecture, design, implementation, pilot, and production. - Owning the technical design and delivery of one core product while contributing to the engineering direction of others. - Collaborating with cross-functional teams to translate business requirements into scalable, production-ready systems. - Defining the technical roadmap and ensuring alignment with Saarthee's long-term product vision. - Dri...

As a Lead Data Engineer at BrandIdea Analytics, you will have the exciting opportunity to shape the core of the analytics ecosystem by building data platforms from the ground up and leading with Python. **Key Responsibilities:** - Lead the design and development of scalable data pipelines using Python, Airflow, dbt, and PySpark. - Integrate multiple data sources including APIs, structured data, and geospatial data. - Build and optimize ETL/ELT workflows and data APIs using FastAPI/Flask. - Drive data quality, governance, and automation across different environments. - Mentor engineers and collaborate closely with Data Science & Product teams. - Contribute to cloud and architecture decisions,...

As a Senior Data Engineer (Scala) at EY Advisory - Data and Analytics, you will have the opportunity to leverage your expertise in the Financial Services domain and Big Data ecosystem to design, build, and maintain scalable data pipelines and solutions. Your role will involve working with Spark, Scala, SQL, and Cloud technologies like GCP, Azure, or AWS to manage and process large datasets efficiently. Additionally, you will be expected to participate in all aspects of the Big Data solution delivery life cycle, from analysis to production deployment and support. **Key Responsibilities:** - Strong knowledge in Spark with a good understanding of the Spark framework and performance tuning - Pro...

As a Snowflake Developer, you will be responsible for the following: - Utilize your 14 years of hands-on experience in data platform support or operations roles - Demonstrate a working knowledge of Snowflake architecture, query optimization, security model, and platform features - Showcase proficiency in SQL to troubleshoot query issues and monitor query performance in Snowflake - Understand Snowflake components such as virtual warehouses, roles, stages, and tasks - Support Snowflake user management, permissions, and role-based access control - Familiarity with scheduling and orchestration tools like Airflow, dbt, or native Snowflake tasks - Collaborate with data engineers and analysts to re...