15386 Airflow Jobs - Page 46

Greetings from Tata Consulting Services!!! Role: Data Engineer with Mandatory Pyspark, Python Knowledge Desired Experience Range: 5 to 10 yrs Location: Pune, Hyderabad, Chennai, Bangalore, Kochi, Bhubaneshwar Notice period: 0 To 30 Days Interview Mode: Virtual Mode Interview Date: 13th Nov 2025 Time Of Interview: 10AM TO 2PM Required Technical Skill Set: Python, Pyspark, Big Query Desired Competencies (Technical/Behavioral Competency) Development, Production Support and delivery of Python, BigQuery, SQL, GCP, Airflow based applications. Resource should have experience in Astronomer DAGS, Python, Pyspark, GCP, BigQuery. Collaborate with cross-functional teams including designers, product owne...

Position: Data Scientist Experience: 4–5 Years (relevant Data Science experience) Location: Pune / Hyderabad / Bangalore Budget: 15LPA Key Skills: Strong hands-on experience in Python (pandas, NumPy, scikit-learn, TensorFlow, PyTorch) or R Solid understanding of Machine Learning , Statistical Modeling , and Data Mining Expertise in Data Visualization (Power BI, Tableau, Matplotlib, Seaborn) Strong SQL and experience with data pipelines (Airflow, Spark, etc.) Proficiency with Cloud ML platforms (AWS SageMaker / Azure ML / Google AI Platform) Nice to Have: Exposure to Deep Learning , NLP , Time Series Analysis Familiarity with MLOps / CI-CD pipelines Certifications or project portfolios in Dat...

Role & responsibilities 1. Technical Leadership & Strategy: Lead the end-to-end architecture, design, and development of advanced AI/ML models (e.g., Deep Learning, NLP, Computer Vision, Predictive Analytics) for healthcare applications. Define the technical vision and roadmap for AI projects, ensuring alignment with business goals and clinical needs. Provide technical guidance and mentorship to a team of data scientists and machine learning engineers while fostering innovation within the team. Stay at the forefront of AI research, evaluating and incorporating state-of-the-art techniques into our solutions. 2. Solution Design & Development: Architect and build ML pipelines for diverse health...

Company Description Elevatoz specializes in creating and managing channel loyalty, customer loyalty, and employee rewards and recognition programs. Our solutions are based on a deep understanding of human behavior, cutting-edge technology, and comprehensive program design to ensure consistent results. We design, launch, and optimize loyalty and engagement programs for channel partners and customers, powered by our user-friendly technology platform. Our expertise spans multiple domains, enabling us to partner with some of the biggest brands in the country. Role Description We are looking for a highly motivated Manager- Data Analytics with strong technical, analytical, and business skills to d...

Role: Data Engineer Experience: 5-10 Years Location: Bannerghatta Road, Bangalore Work Mode: Onsite (5 days a week) Notice Period: Immediate Joiners Preferred Key Skills: Databricks, PySpark Responsibilities: Develop, optimize, and maintain scalable data pipelines using Databricks and PySpark . Manage data integration, transformation, and loading from multiple sources. Ensure data quality, reliability, and performance across platforms. Collaborate with data scientists and analysts to support analytical models and dashboards. Troubleshoot issues and provide production-level data support. Requirements: 510 years of hands-on experience in data engineering . Strong programming experience with Py...

Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc with coding skills in Python PySpark and SQL. Works independently and demonstrates proficiency in at least one domain related to data with a solid understanding of SCD concepts and data warehousing principles. Outcomes: Collaborate closely with data analysts data scientists and other stakeholders to ensure data accessibility quality and security across various data sources.rnDesign develop and maintain data pipelines...

Role & Responsibilities We are seeking a highly skilled Senior DevOps Engineer with 8+ years of hands-on experience in designing, automating, and optimizing cloud-native solutions on AWS. AWS and Linux expertise are mandatory. The ideal candidate will have strong experience across databases, automation, CI/CD, containers, and observability, with the ability to build and scale secure, reliable cloud environments. Key Responsibilities Cloud & Infrastructure as Code (IaC)- Architect and manage AWS environments ensuring scalability, security, and high availability. Implement infrastructure automation using Terraform, CloudFormation, and Ansible. Configure VPC Peering, Transit Gateway, and Privat...

We are looking for a passionate and detail-oriented Software Engineer to join our Big Data / Data Warehouse (DWH) / ETL team. The ideal candidate should have a strong understanding of data systems, data processing pipelines, and an interest in large-scale data analytics. Fresh graduates with relevant academic projects or training in data engineering are also welcome to apply. Responsibilities Design, develop, and maintain ETL pipelines for efficient data extraction, transformation, and loading. Work with Big Data technologies (Hadoop, Spark, Hive, etc. ) to process and analyse large datasets. Develop and maintain data warehouse models and ensure data integrity and consistency. Optimise and t...

Your Opportunity As an architect, you will get the opportunity to grow in the field of high-performance computing, complex system design and low-level optimizations for better cost of ownership. Roles and Responsibility As a Software Architect, you will be responsible for designing and implementing High performance computing software solutions for our organization. You will work closely with cross-functional teams, including software engineers, product managers, and business stakeholders, to understand requirements and translate them into architectural/software designs that meet business needs. You will be coding and developing quick prototypes to establish your design with real code and dat...

Description: Skills: Proficiency in SQL is a must. PL/SQL to understand integration SP part. Experience in PostgreSQL is must. Basic knowledge of Google Cloud Composer ( or Apache Airflow). Composer is managed GCP service for Apache Airflow. All pipelines are orchestrated and scheduled through Composer GCP basics-high level understanding of using GCP UI and services like Cloud SQL PostgreSQL Cloud Composer Cloud Storage Dataproc Airlfow DAGs are written in Python basic knowledge of Python code for DAGs Dataproc is Managed Spark in GCP so a bit of PySpark knowledge is also nice to have. Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade :C Level :To Be ...

Description: Objective:Build and maintain data pipelines and infrastructure to support data storage and processing. Key Responsibilities: -Design develop and optimize data pipelines using Azure Databricks. -Ensure data reliability scalability and quality. -Integrate data sources into a unified platform. -Monitor and troubleshoot data workflows. Skills Required: -Expertise in Azure Databricks Spark and big data tools. -Strong programming skills (e.g. Python Scala SQL). -Knowledge of ETL processes and cloud technologies. -Familiarity with CI/CD practices for data pipelines Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade :C Level :To Be Defined Named J...

Python Developer Exp- 5-9yrs Max salary 25lpa-28lpa Pune and Hyd (Hybrid -WFO) Fixed rate - FTE Python Data Integration Engineer :Are you a seasoned data engineer with a passion for both hands-on technical work? Do you thrive in an environment that values innovation, collaboration, and cutting-edge technologies? We are seeking an experienced Data Integration engineer to spearhead our data integration strategies and initiatives. The ideal candidate will possess deep technical expertise in Python programming, Snowflake data warehousing, AWS cloud services, Kubernetes (EKS), CI/CD methodologies, Apache Airflow, dbt, Kafka for real-time data streaming, and API development. This role is pivotal i...

Description Key job responsibilities include, but are not limited to Become involved with day-to-day operations in Production Control. Understand the workflows and be able to troubleshoot issues and define new workflows in PCS and Control-M Design Airflow infrastructure according to the industry best practices. Work with System Administrators, Platform Engineers, Network, Firewall and Monitoring teams on the implementation of the infrastructure. Document the system's specifics, define, and implement administrative controls and policies of Airflow. Work with the development teams to identify the groups of jobs to migrate to airflow. Assist the development teams with the migrationsdefine DAGs,...

Python +SQL developer Python Data Integration Engineer Role and Responsibilities:As the Data Integration Engineer, you will play a pivotal role in shaping the future of our data integration engineering initiatives. You will remain actively involved in the technical aspects of the projects. Your responsibilities will includeHands-On ContributionContinue to be hands-on with data integration engineering tasks, including data pipeline development, EL processes, and data integration. Be the go-to expert for complex technical challenges. Integrations ArchitectureDesign and implement scalable and efficient data integration architectures that meet business requirements. Ensure data integrity, qualit...

JR REQ---Data Engineer(Pyspark, Big mailto:data)--4to8year---hyd----hemanth.karanam@tcs.com----TCS C2H ---900000

We are seeking a skilled GCP Data Engineer / Developer to design, build, and optimize scalable data pipelines and applications on Google Cloud Platform (GCP) . The ideal candidate will have hands-on experience with BigQuery, PySpark, and Airflow DAGs , with added advantage if they possess Java and ReactJS development experience for end-to-end data and application development. Key Responsibilities: Design, develop, and maintain data ingestion and transformation pipelines on GCP using BigQuery, PySpark , and Airflow DAGs . Implement data processing and analytics workflows aligned with performance, scalability, and reliability standards. Collaborate with cross-functional teams (data architects,...

Data engineer Skills Required: Strong proficiency in Pyspark , Scala , and Python Experience with AWS Glue Experience Required: Minimum 5 years of relevant experience Location: Available across all UST locations Notice Period: Immediate joiners (Candidates available to join by 31st January 2025 ) SO - 22978624 Location - Chandigarh,Dadra & Nagar Haveli,Daman,Diu,Goa,Haveli,Jammu,Lakshadweep,Nagar,New Delhi,Puducherry,Sikkim

Should be capable of developing/configuring data pipelines in a variety of platforms and technologies Possess the following technical skills SQL Python Pyspark Hive ETL Unix Control-M (or similar scheduling tools) Can demonstrate strong experience in writing complex SQL queries to perform data analysis on Databases SQL server Oracle HIVE etc. Experience with GCP (particularly Airflow Dataproc Big Query) is an advantage Have experience with creating solutions which power AI/ML models and generative AI Ability to work independently on specialized assignments within the context of project deliverables Take ownership of providing solutions and tools that iteratively increase engineering efficien...

Job Description:- We are seeking a skilled and detail-oriented BIM Modeler (Mechanical) to join our design and engineering team in Bangalore. The ideal candidate will have strong expertise in CAD to BIM conversion, 3D modeling, and MEP coordination, with hands-on experience in Revit and other BIM tools. The role involves producing high-quality BIM models and drawings in alignment with project standards and client requirements . Key Skills (Mechanical BIM Modelling):- - Proficient in HVAC system modeling including ducts, chillers, diffusers, AHUs, and piping. - Experience in 3D modeling and coordination of mechanical systems in Revit or similar BIM tools. - Knowledge of airflow design, temper...

Client expectation apart from JD Longer AWS data engineering experience (glue, spark, ECR ECS docker), python, pyspark, hudi/iceberg/Terraform, Kafka. Java in early career would be a great addition but not a prio. (for the OOP part and java connectors).

An Amazing Career Opportunity for Senior AI/ML Engineer Location: Chennai, India (Hybrid) Job ID: 42889 Profile Summary: A rewarding career at HID Global beckons you! We are looking for a Senior AI/ML Engineer , who is responsible for designing, developing, and deploying advanced AI/ML solutions to solve complex business challenges. This role requires expertise in machine learning, deep learning, MLOps, and AI model optimization , with a focus on building scalable, high-performance AI systems. As a Lead AI/ML Engineer , you will work closely with data engineers, software developers, and business stakeholders to integrate AI-driven insights into real-world applications. You will be responsibl...

Job Title: Machine Learning Engineer Location: Bengaluru (Hybrid) Experience: 2-6 years About Wissen Infotech Wissen Infotech has been a trusted leader in the IT Services industry for over 25 years, delivering high-quality solutions to a global clientele. Within Wissen, the AI Center of Excellence (AI-CoE) was conceptualized to drive cutting-edge research and innovation, enabling us to build our own products and intellectual property. This team focuses on solving complex business challenges using AI while setting new benchmarks for reliable and scalable AI solutions. Position Overview We are seeking a passionate Machine Learning Engineer to join our AI-CoE team. This is a unique opportunity ...

This is an exciting opportunity to be part of a team that will be instrumental in crafting the future of how we do business with the commercial customer at the center of everything we do. Are you passionate about applying technology to solve business problems? Are you someone who doesn’t just do exactly what you’re told, but proactively seeks to understand and solve underlying pain points? Are you motivated to apply your skills to build the solutions that will increase the productivity of our commercial customers? This specific Product Manager role is a rare opportunity to work on a customer-facing, revenue-generating application. Are you up for the challenge? If so, then join our team as a ...

You will work on ingesting, transforming, and analyzing large datasets to support the Enterprise Data Platform on Google Cloud Platform (GCP). Experience with large scale solution and operationalization of data lakes, data warehouses, and analytics platforms on Google Cloud Platform or other cloud environments is a must. Experience in analyzing complex data, organizing raw data, and integrating massive datasets from multiple data sources to build analytical domains and reusable data products. 3+ years of cloud experience (GCP preferred) with solutions designed and implemented at production scale. Experience working in GCP native (or equivalent) services like Big Query, Google Cloud Storage, ...

3+ years of cloud experience (GCP preferred) with solutions designed and implemented at production scale. Experience working in GCP native (or equivalent) services like Big Query, Google Cloud Storage, PubSub, Dataflow, Dataproc, Cloud Build, etc. Experience working with Airflow for scheduling and orchestration of data pipelines. Experience working with Terraform to provision Infrastructure as Code. 2 + years professional development experience in Java or Python. Bachelor’s degree in computer science or related scientific field. Bachelor’s degree in computer science or related scientific field. Work on a Central Engineering Portfolio Team based in Chennai to deliver curated data products and...