15348 Airflow Jobs - Page 29

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Data Engineer Job Description Develop and maintain scalable data pipelines using tools like Apache Kafka, Apache Spark, or AWS Glue to ingest, process, and transform large datasets from various sources, ensuring efficient data flow and processing. Design and implement data models and schemas in data warehouses (e.g., Amazon Redshift, Google BigQuery) and data lakes (e.g., AWS S3, Azure Data Lake) to support analytics and reporting needs. Collaborate with data scie...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Core Responsibilities Of a Data Engineer In D&A Data Pipeline Design & Development Data Engineers are responsible for designing and building robust, scalable, and high-quality data pipelines that support analytics and reporting needs. This includes: ETL/ELT development using tools like Azure Data Factory, Databricks, and Snowflake. Integration of structured and unstructured data from various sources into data lakes and warehouses. Cloud Platform Engineering They o...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Data Engineer Job Description Develop and maintain scalable data pipelines using tools like Apache Kafka, Apache Spark, or AWS Glue to ingest, process, and transform large datasets from various sources, ensuring efficient data flow and processing. Design and implement data models and schemas in data warehouses (e.g., Amazon Redshift, Google BigQuery) and data lakes (e.g., AWS S3, Azure Data Lake) to support analytics and reporting needs. Collaborate with data scie...

Dear Candidates, We have urgent openings for Big data engineer - CMM L5 Client- Permanent position Location-Chennai/Bangalore-Hybrid mode Exp-4-15yrs NP- Immediate to 20days max(Serving NP only) Job Description: Primary Skills: Hadoop, Hive, Python, Spark, SQL, Airflow... Through and hands on knowledge on Python and Spark Programming and writing efficient, reusable, and reliable Python code. Very good proficiency in Hadoop, Python, Airflow, SQL and Spark programming. Implementation experience of Spark Core, Spark SQL and Spark Streaming Working with Spark in combination Hadoop Ecosystem. if anyone interested pls share your resume to Manasa@skyonn.com

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! Role: Senior Data Engineer Experience Level: 4 to 6 Years Work location: Mumbai, Bangalore, Trivandrum (Hybrid Only) Required Skills Bachelor’s degree in Computer Science or similar field or equivalent work experience. 3+ years of experience on Data Warehousing, Data Engineering or Data...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Data Engineer Job Description Develop and maintain scalable data pipelines using tools like Apache Kafka, Apache Spark, or AWS Glue to ingest, process, and transform large datasets from various sources, ensuring efficient data flow and processing. Design and implement data models and schemas in data warehouses (e.g., Amazon Redshift, Google BigQuery) and data lakes (e.g., AWS S3, Azure Data Lake) to support analytics and reporting needs. Collaborate with data scie...

We are seeking a highly skilled and experienced AWS Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS, Python, & Spark. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and implement scalable data pipelines for real-time, batch, structured, and unstructured data . Design and develop solutions using AWS services such as EMR, Kinesis, Glue, Athena, S3, CloudFormation, API Gateway . Work extensively with streaming platforms like Kafka, Flink, Spark Streaming . Develop and optimize data ingestion workflows using Apache NiFi,...

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Associate Job Description & Summary At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data tra...

SUMMARY As a Associate SRE, you’ll support the design, deployment, and monitoring of cloud and data systems to ensure reliability, scalability, and uptime. You’ll collaborate with developers and infrastructure teams to maintain production environments and automate routine operations. Responsibilities: Support CI/CD pipeline setup and basic monitoring systems for applications and data pipelines. Assist in automating deployments using tools like Jenkins, Git, and Terraform. Monitor performance metrics and escalate incidents to senior engineers. Maintain documentation and participate in root cause analysis for minor incidents. Work under supervision to implement application and data reliability...

SUMMARY Job Description: You will own critical reliability functions across cloud and data platforms. This role involves improving application availability, automating deployments, and ensuring data systems meet SLAs through observability and proactive performance monitoring. Responsibilities: Design, automate, and maintain CI/CD pipelines for application and data environments. Implement Infrastructure as Code (IaC) using Terraform or Ansible. Work with Data Engineering and DevOps teams to ensure system uptime and reliability. Conduct incident response, post-mortems, and continuous improvement. Create dashboards and alerts for performance metrics using Grafana, CloudWatch, or Dynatrace. Requ...

SUMMARY Job Description: As a Lead SRE, you will architect, implement, and manage cloud and data reliability strategies across large-scale applications and client environments. This role emphasizes system scalability, resilience, automation, and mentoring junior engineers. Responsibilities: Lead design and implementation of reliability frameworks for applications and data pipelines. Architect and secure scalable cloud infrastructure (AWS / Azure). Drive incident management, service availability, and disaster recovery planning. Automate provisioning and configuration using Terraform, Ansible, or CloudFormation. Oversee monitoring systems, SLO/SLI tracking, and cost optimization. Mentor junior...

Ascendeum is looking for veterans with extensive hands-on experience in the field of data engineering to build cutting-edge solutions for large-scale data extraction, processing, storage, and retrieval. About Us: We provide AdTech strategy consulting to leading Internet websites and apps globally hosting over 200 million monthly worldwide audiences. Since 2015, our team of consultants and engineers have been consistently delivering intelligent solutions that enable enterprise-level websites and apps to maximize their digital advertising returns. Job Responsibilities: Understand long-term and short-term business requirements to precisely match them with the capabilities of different distribut...

Ascendeum is looking for veterans with extensive hands-on experience in the field of data engineering to build cutting-edge solutions for large-scale data extraction, processing, storage, and retrieval. About Us: We provide AdTech strategy consulting to leading Internet websites and apps globally hosting over 200 million monthly worldwide audiences. Since 2015, our team of consultants and engineers have been consistently delivering intelligent solutions that enable enterprise-level websites and apps to maximize their digital advertising returns. Job Responsibilities: Understand long-term and short-term business requirements to precisely match them with the capabilities of different distribut...

Senior Backend Engineer Location: India - Remote Type: Full-time Compensation: ₹20-50 LPA basis experience We are hiring a Senior Backend Engineer on behalf of a fast-growing SaaS startup that is building intelligent inventory and forecasting systems for modern e-commerce businesses. As one of the first backend hires, you will work closely with the CTO to design, scale, and optimize a high-performance, data-intensive platform already used by hundreds of merchants. This is a hands-on role with significant ownership, technical depth, and product impact. Responsibilities Architect, design, and implement scalable backend systems and APIs handling millions of data points Improve reliability, obse...

SUMMARY Job Summary As a Data SRE Lead, you will architect and oversee end-to-end reliability strategies for data infrastructure. You’ll work with cloud, data engineering, and analytics teams to ensure that critical data services are secure, automated, observable, and resilient. Responsibilities Lead design and automation of resilient data platforms across multi-cloud environments. Drive data reliability frameworks (SLO/SLI metrics, incident prevention, SLA tracking). Oversee monitoring, logging, and cost optimization for big-data environments. Manage data platform scalability, versioning, and fault tolerance. Build recovery and rollback mechanisms for critical data pipelines. Guide teams in...

🚀 We’re Hiring – Data Engineer (4+ Yrs Experience) - FTE 📍 Location: Chennai (Onsite) 🕒 Notice Period: Immediate Joiners Only Key Skills: ☁️ Primary Cloud Expertise: GCP (preferred) or Azure 🧠 SQL – advanced & scenario-based problem solving 🐍 Python – strong hands-on experience 🔧 GCP Services: Cloud Functions, BigQuery, Airflow, Pub/Sub 👉 Note: AWS experience not mandatory Share your resume at [rajesh@reveilletechnologies.com]./.

Data Engineer As a Data Engineer, you will work across a global team of analysts and data operation associates to develop new data technologies for Data Operations at PitchBook. We’re looking for a person who can combine proficiency in data engineering technologies (SQL, Python, Data Warehousing & ETL tools, Power BI) with an operational mindset and deliver powerful new data sourcing, automation and efficiency tools to the team. You will become a master PitchBook’s data operations processes and methodologies, establishing the process knowledge necessary to develop effective data solutions. What you'll do: Extracting data from various sources, including web scraping, API requests, etc. Buildi...

GCP Data Engineer We are seeking a skilled GCP Data Engineer with 5 to 10 years of experience in cloud-based data engineering, including minimum 5+ years of hands-on work with GCP services such as BigQuery, Dataflow, Airflow , and Python . Key Responsibilities Design, develop, and deploy scalable IoT data pipelines using GCP services. Build and optimize data workflows using BigQuery , Dataflow , and Airflow . Write clean, reusable, and maintainable Python code for data processing tasks. Monitor and troubleshoot data ingestion and transformation processes to ensure reliability. Collaborate with cross-functional teams to integrate IoT data with analytics and monitoring platforms. Implement sec...

Job description below. 5+ years of experience in the data engineering field is preferred 3+ years of Hands-on experience in GCP cloud data implementation suite such as Big Query, Pub Sub, Data Flow/Apache Beam, Airflow/Composer, Cloud Storage, Strong experience and understanding of very large-scale data architecture, solutioning, and operationalization of data warehouses, data lakes, and analytics platforms. Hands on Strong Experience in the below technology 1. GBQ Query 2. Python 3. Apache Airflow 4. SQL (BigQuery preferred) Extensive hands-on experience working with data using SQL and Python Cloud Functions. Comparable skills in AWS and other cloud Big Data Engineering space is considered....

Lead Data Engineer – Python Ecosystem 📍 Chennai, India | Hybrid We’re growing our technology team at BrandIdea Analytics — a pioneer in spatial and behavioral analytics that helps businesses see their markets in entirely new ways. If you’re passionate about building data platforms from the ground up and leading with Python, this is your opportunity to shape the core of our analytics ecosystem. What You’ll Do Lead design and development of scalable data pipelines (Python, Airflow, dbt, PySpark). Integrate multiple data sources – APIs, structured & geospatial data. Build and optimize ETL/ELT workflows and data APIs (FastAPI/Flask). Drive data quality, governance, and automation across environm...

ABOUT AMGEN Amgen harnesses the best of biology and technology to fight the world’s toughest diseases, and make people’s lives easier, fuller and longer. We discover, develop, manufacture and deliver innovative medicines to help millions of patients. Amgen helped establish the biotechnology industry more than 40 years ago and remains on the cutting-edge of innovation, using technology and human genetic data to push beyond what’s known today. ABOUT THE ROLE Role Description: As the Data Scientist at Amgen, you will be responsible for developing and deploying advanced machine learning, operational research, semantic analysis, and statistical methods to uncover structure in large data sets. Thi...

Join Amgen’s Mission of Serving Patients At Amgen, if you feel like you’re part of something bigger, it’s because you are. Our shared mission—to serve patients living with serious illnesses—drives all that we do. Since 1980, we’ve helped pioneer the world of biotech in our fight against the world’s toughest diseases. With our focus on four therapeutic areas –Oncology, Inflammation, General Medicine, and Rare Disease– we reach millions of patients each year. As a member of the Amgen team, you’ll help make a lasting impact on the lives of patients as we research, manufacture, and deliver innovative medicines to help people live longer, fuller happier lives. Our award-winning culture is collabo...

Role description Job Description We are seeking an experienced Snowflake Database Administrator DBA to join our team The ideal candidate will have a deep understanding of Data Warehousing fundamentals and concepts with a proven track record in Snowflake administration Key Responsibilities Database Management Oversee and manage Snowflake databases including user management security management performance tuning high availability space management and backuprecovery processes Automation Implement and support automation for production deployments in Snowflake Automate database administration tasks using PLSQL shell scripts and other relevant scripting languages Troubleshooting Diagnose and resol...

Role: Bigdata Engineer Location: Banglore/ Chennai Duration: Full time Primary Skill: Hadoop, hive, Airflow, Python, Spark, SQL Job Description: Very good proficiency in Python and Spark programming. Pandas: Experience with data manipulation and analysis using Pandas. Implementation experience of Spark Core, Spark SQL and Spark Streaming Working with Spark in combination Hadoop Ecosystem Design and implementation of low-latency, high-availability, and performance applications. Should lead and guide a team of junior Python developers. As Sr Python Developer responsibilities include coding, testing, debugging, and troubleshooting throughout the application development process. Performance tuni...

About The Role: Grade Level (for internal use): 11 About The Role: We are seeking a Lead Machine Learning and MLOps Engineer focused on Generative AI to join our ML team within the Data Science Center of Excellence at S&P Global. In this role, you will lead engineering activities to build production-grade generative AI solutions and play a pivotal role in implementing machine learning engineering operations to ensure seamless deployment, monitoring, and management of our machine learning models and data pipelines. The Team: You will work closely with a world-class AI and ML team comprised of experts in AI and ML modeling, MLOps engineers, data science, and data engineering. Your contribution...