10830 Airflow Jobs - Page 11

Hiring GenAI Developer (NLP, LLMs, Python, TensorFlow/PyTorch, AWS) & GCP Data Engineer (BigQuery, Dataflow, SQL, Airflow). Hyderabad/Bangalore. Immediate joiners preferred. Apply now!

Overview GCP, Big Query, Composer, Python, Pyspark, Dataproc, Airflow, SQL Key Responsibilities: Design, develop, and maintain robust and scalable data pipelines using GCP services such as Dataflow, Dataproc, BigQuery, Pub/Sub, and Cloud Storage. Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver efficient solutions. Implement ETL/ELT workflows, data ingestion, transformation, and validation processes. Optimize performance and cost-efficiency of data processing jobs and cloud resources. Troubleshoot and resolve data pipeline issues and production incidents. Ensure data quality, governance, and compliance standards are met. Automate data w...

Job Summary: We are seeking a skilled GCP Data Engineer to design, develop, and manage large-scale data pipelines and cloud-based data solutions on Google Cloud Platform (GCP) . The role requires expertise in data engineering, ETL/ELT processes, and GCP services to deliver scalable, reliable, and high-performance data platforms that support analytics and business intelligence. Key Responsibilities: Design, build, and maintain data pipelines and ETL/ELT workflows using GCP services. Develop scalable data architectures leveraging BigQuery, Dataflow, Pub/Sub, Dataproc, and Cloud Storage . Collaborate with data analysts, data scientists, and business teams to understand requirements and deliver ...

Job Title: Snowflake & Matillion Data Architect 📍 Location: Remote 🕒 Work Mode: 100% Remote (Must have EST time zone overlap) 📅 Employment Type: Long-Term Contract 🚫 Dual Employment: Not permitted — must be terminated before onboarding 🔧 Key Responsibilities Design and implement end-to-end data architecture using Snowflake and Matillion for ingestion, transformation, and storage Define and maintain best practices in data modeling (e.g., dimensional, normalized, star/snowflake schemas) Lead the adoption and scaling of modern data stack tools aligned with business objectives Collaborate with data engineers, analysts, and business teams to define data requirements and build reliable data pipeli...

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions. Job Description At PwC - AC, as an AWS Architect / Manager, the candidate will interact with Offshore Manager/ Onsite ...

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions. At PwC - AC, as an AWS Architect / Manager, the candidate will interact with Offshore Manager/ Onsite Business Analyst...

Investigate, research, review, evaluate, and elicit requirements. Conduct risks assessment and provide mitigation recommendations. Perform functional data or back-end test executions by writing SQL and automation scripts. Facilitate and support User Acceptance Test and Production Verification test management. Conduct test and defect management, including test reporting. Implement suggested quality test framework and submit to the review and audit activities. Experiences Experience with analysing big data or data warehouse. Experience in data migration and ETL (extract, transform and load) process. Good to have knowledge and/or experience with Cloud Platforms. Experience with automation testi...

Hiring for Snowflake_Bangalore Design, develop, and maintain scalable data pipelines using Snowflake . Implement ETL/ELT processes to ingest data from various sources (e.g., APIs, databases, flat . Optimize Snowflake performance using best practices (e.g., clustering keys, caching, virtual warehouse sizing). Design and develop data models (star/snowflake schema) that support business intelligence and analytics needs. Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements. Implement data quality checks , monitoring, and alerting for data pipelines. Use tools such as dbt , Apache Airflow , Matillion , or other orchestration tools for workflow...

Citrix Observability with Splunk ITSI: Expertise in utilizing Citrix observability data with Splunk IT Service Intelligence for comprehensive monitoring and analytics of Citrix environments. VDI and VM Observability: Proficient in monitoring Virtual Desktop Infrastructure (VDI) and Virtual Machines (VMs) for performance, reliability, and scalability. OpenTelemetry, Kafka, and Splunk/Grafana: Hands-on experience with OpenTelemetry for unified telemetry, Kafka for real-time data ingestion, and Splunk/Grafana for powerful data visualization and alerting. Data Science Machine Learning: Proficient in Python, TensorFlow, PyTorch, Scikit-learn, Pandas, NumPy for developing machine learning models f...

The ideal candidate for this role should possess a strong blend of MLOps and ML engineering skills. Technological Stack: Python, AWS services (ECS, Batch Compute, S3, Timestream, IAM, CloudWatch, Athena), Airflow for orchestration, MLflow for model versioning, GitLab CI, and eventually, AWS SageMaker pipelines. Data Engineering Skills: Expertise in pipeline monitoring, including getting data from Bulk CDM, investigating query issues, and sending output data to ALTO. ML Engineering Skills: Experience with debugging forecast and clustering models, setting up and automating CI/CD for ML workflows, developing and deploying end-to-end ML pipelines (data preparation, training, validation, deployme...

About The Role We are seeking an experienced and passionate Senior Database Engineer to join our technology team. The ideal candidate will have strong expertise in SQL development, PostgreSQL administration, and Python scripting to support data-intensive applications. The role requires designing and optimizing databases, building data pipelines, and ensuring seamless migration of customer data into new product hosted on AWS cloud. Key Responsibilities Design, implement, and optimize PostgreSQL databases for scalability, security, and performance. Develop and maintain complex SQL queries, procedures, views, and triggers to support application requirements. Write Python or sql scripts for data...

About The Role We are seeking an experienced and passionate Database Engineer to join our technology team. The ideal candidate will have strong expertise in SQL development, PostgreSQL administration, and Python scripting to support data-intensive applications. The role requires designing and optimizing databases, building data pipelines, and ensuring seamless migration of customer data into new product hosted on AWS cloud. Key Responsibilities Design, implement, and optimize PostgreSQL databases for scalability, security, and performance. Develop and maintain complex SQL queries, procedures, views, and triggers to support application requirements. Write Python or sql scripts for database au...

• Proficient in Git for version control , SQL & exp with DB technologies,data pipeline tools such as Apache Airflow,data migration and validation techniques.Strong knowledge of Apache Spark for data processing and transformation,data governance.

Hands-On | Micro Data Lakes | Enterprise Data Strategy Are you a hands-on Data Architect who thrives on solving complex data problems across structured and unstructured sources Do you enjoy designing micro data lakes and driving enterprise-wide data strategy If so, we want to hear from you. What You Will Do Design and build micro data lakes tailored to lending domain Define and implement enterprise data strategies including modelling, lineage, and governance Architect and build robust data pipelines for batch and real-time data ingestion Develop strategies for extracting, transforming, and storing data from APIs, PDFs, logs, databases, and more Establish best practices for data quality, meta...

We are united in our mission to make a positive impact on healthcare. Join Us! South Florida Business Journal, Best Places to Work 2024 Inc. 5000 Fastest-Growing Private Companies in America 2024 2024 Black Book Awards, ranked #1 EHR in 11 Specialties 2024 Spring Digital Health Awards, “Web-based Digital Health” category for EMA Health Records (Gold) 2024 Stevie American Business Award (Silver), New Product and Service: Health Technology Solution (Klara) Who We Are We Are Modernizing Medicine (WAMM)! We’re a team of bright, passionate, and positive problem-solvers on a mission to place doctors and patients at the center of care through an intelligent, specialty-specific cloud platform. Our v...

What you’ll do: Design, build, and maintain scalable data pipelines and platforms Implement DevOps practices for data engineering workflows Manage AWS infrastructure, Snowflake, dbt Core, Apache Airflow, Atlan Drive CI/CD quality using GitHub Actions, Terraform, SonarQube, Docker, Kubernetes Ensure data governance, role-based access, cost optimization, and healthcare compliance Collaborate with data, dev, and product teams to deliver high-quality, reliable platforms What we need: • 5+ years of experience in Data Engineering + DevOps • Strong hands-on experience with Python, AWS, Snowflake, Airflow, Terraform, dbt Core • Familiarity with Docker, Kubernetes, GitOps, CI/CD pipelines • Knowledge...

Position Description: Employees in this job function are responsible for developing intelligent programs, cognitive applications and algorithms for data analysis and automation, leveraging various AI techniques such as deep learning, generative AI, natural language processing, image processing, cognitive automation, intelligent process automation, reinforcement learning, virtual assistants and specialized programming Key Responsibilities: • Agile personality and AI Engineering Craftsmanship mindset for leading and influencing an Agile team particularly for Enterprise AI Technology projects • Should possess Capabilities to contribute to AI application development and model deployment project ...

Job Summary We are looking for a skilled and motivated Data Scientist to join our team as an individual contributor . The ideal candidate should have at least two years of experience in data science or ML engineering, with hands-on expertise in Databricks , PySpark , and SQL . You’ll be responsible for owning and delivering scalable machine learning solutions end-to-end—from exploration to production—working independently and collaborating closely with business and engineering teams. Key Responsibilities Own and deliver the complete lifecycle of machine learning projects, from data exploration and feature engineering to model deployment and monitoring. Develop, optimize, and maintain ML and ...

Analysis and problem-solving capabilities. • Degree in Maths/Physics/Statistics/similar discipline or previous experience of a similar role. • Ability to interpret data, recognize problems as they arise, suggest and implement appropriate actions. • Ability to communicate clearly, both verbally and via detailed specifications and reports. • Excellent organizational skills and capability of working to tight deadlines. • Ability to work to a high level of accuracy. • Experience in testing applications using Python and Pyspark in a Hadoop environment with HDFS. • Experience in AWS services S3, Lambdas, Managed Airflow (MWAA), and EMR Serverless. • Familiarity with agile ways of working and data ...

We're hiring a Software Engineer to join our growing team. You'll be part of a young, ambitious, and collaborative engineering culture. We ship new features weekly, work across exciting technologies, and constantly push ourselves to build products that are efficient, scalable, and impactful. If you're eager to learn, grow, and make real contributions from day one, this is the place for you. Responsibilities Build and maintain applications using Node.js, Golang, Java, and other modern technologies. Create and manage data pipelines with ETL tools (e. g., Apache Airflow), connecting to MongoDB and SQL databases. Work with Kafka or similar message queues for real-time data. Partner with DevOps t...

We're hiring a Senior Software Engineer to take ownership of key systems and guide our engineering growth. You'll play a crucial role in designing, scaling, and optimizing our systems while mentoring teammates and shaping our engineering culture. This is your chance to work across exciting technologies, drive impact, and help us ship smarter and faster. Responsibilities Architect, build, and maintain applications using Node.js, Golang, Java, and other modern technologies. Design and manage data pipelines with ETL tools (e. g., Apache Airflow), connecting to MongoDB and SQL databases. Own and optimize Kafka or similar message queue systems for real-time data. Partner with DevOps to manage Kub...

We're hiring a Software Engineer at Refold.ai . This is a rare opportunity to build and scale cutting-edge AI-first products. You'll work closely with a talented team to design, develop, and ship features that power our growth. This role sits at the intersection of software engineering, data engineering, and AI, requiring someone who can think critically while executing with excellence. Responsibilities Build and maintain applications using Node.js, Golang, Java, and other technologies. Design and manage data pipelines with ETL tools (e. g., Apache Airflow), integrating MongoDB and SQL databases. Implement real-time systems using Kafka or similar messaging queues. Collaborate with AI enginee...

We are seeking a Data Engineer to design and maintain scalable data pipelines that power finance-related analytics and reporting. This role involves partnering closely with finance, product, and business teams to translate requirements into robust technical solutions while ensuring data accuracy, consistency, and performance. You will work with modern data technologies, including SQL, Python, Spark, Airflow, and AWS cloud services to build efficient workflows, optimize large-scale datasets, and deliver business-critical insights. Responsibilities Design, build, and maintain scalable data pipelines for finance-related datasets. Partner with finance stakeholders (product and business teams) to...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Associate Job Description & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data tra...

Sia is a next-generation, global management consulting group. Founded in 1999, we were born digital. Today our strategy and management capabilities are augmented by data science, enhanced by creativity and driven by responsibility. We’re optimists for change and we help clients initiate, navigate and benefit from transformation. We believe optimism is a force multiplier, helping clients to mitigate downside and maximize opportunity. With expertise across a broad range of sectors and services, our 3,000 consultants serve clients worldwide from 48 locations in 19 countries. Our expertise delivers results. Our optimism transforms outcomes. Strategy & Management Consulting Sia’s Strategy & Manag...