10894 Airflow Jobs - Page 49

Experience: 5-9 Years Location: Bangalore,Chennai,Gurgaon,Kolkata,Pune Work Mode: Hybrid Mandatory Skills: Azure Datafactory,Snowflake,ETL,Sql,python,pyspark. Primary Roles And Responsibilities Developing Modern Data Warehouse solutions using Snowflake, Databricks and ADF. Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in the reporting layer and develop a data model to fulfill reporting needs Help joiner team members to resolve issues and technical cha...

Role : Backend Engineer Location : Hyderabad, India (Hybrid Knowledge City) Experience : 7 - 10 years Employment : Full - Time with Crunchyroll Job Description: We are looking for a highly skilled Backend Engineer with strong expertise in Python and cloud platforms. The role involves building scalable backend services, integrating with distributed systems, and supporting data-driven workflows. Key Responsibilities: Design and develop backend services and APIs using Python. Build and optimize applications for performance, scalability, and reliability. Work with cloud platforms (GCP/AWS) to deploy and manage services. Collaborate with cross-functional teams to deliver high-quality, production-...

We are seeking a skilled Data Scientist with strong Python development expertise to join our dynamic and fast-growing team in Hyderabad. This is a hands-on role that involves working across the entire data pipeline from data wrangling to modeling, visualization, and client engagement. If you're passionate about using data to solve real-world problems and enjoy taking ownership of your work, we’d love to hear from you. Key Responsibilities Collect, clean, and analyze both structured and unstructured datasets Perform Exploratory Data Analysis (EDA) to uncover insights and drive data-driven decisions Build, train, and deploy machine learning and statistical models Develop efficient, reusable, a...

Job Summary Synechron is seeking a highly skilled and experienced Senior Data Engineer to join our innovative analytics team in Bangalore. The primary purpose of this role is to design, develop, and maintain scalable data pipelines and architectures that empower data-driven decision making and advanced analytics initiatives. As a critical contributor within our data ecosystem, you will enable the organization to harness large, complex datasets efficiently, supporting strategic business objectives and ensuring high standards of data quality, security, and performance. Your expertise will directly contribute to building robust, efficient, and secure data solutions that drive business value acr...

Decision Analytics EXL (NASDAQ:EXLS) is a leading operations management and analytics company that helps businesses enhance growth and profitability in the face of relentless competition and continuous disruption. Using our proprietary, award-winning Business EXLerator Framework™, which integrates analytics, automation, benchmarking, BPO, consulting, industry best practices and technology platforms, we look deeper to help companies improve global operations, enhance data-driven insights, increase customer satisfaction, and manage risk and compliance. EXL serves the insurance, healthcare, banking and financial services, utilities, travel, transportation and logistics industries. Headquartered...

Company Overview Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milest...

7+ Years of expertise required in below skillsets: Data Engineering, Python, SQL, PySpark, Cloud (Any), Airflow, DevOps & Orchestration, Agile, Scrum, Data Pipeline Development, Stakeholder Management, Banking/ Telecom Domain Experience.

Company Description At WorkFox Solutions, we bridge the gap between top talent and forward-thinking companies, helping both grow and thrive. We offer end-to-end recruitment services, executive search, talent acquisition strategies, HR consulting, and personalized career counseling. Our approach combines human intuition with data-driven insights to build trust, transparency, and long-term partnerships. Whether you're an employer searching for the perfect fit or a job seeker looking for your next opportunity, we are here to support your journey. Role Description We are seeking a Senior Consultant – AI & Machine Learning with 8–10 years of experience to provide strategic guidance, technical exp...

LLM Orchestration & RAG Development (LangChain/LlamaIndex/PydanticAI Focus) Architect complex LangChain pipelines for multi-agent financial workflows Build production RAG systems using LlamaIndex for financial document retrieval Implement agents with strong type safety and structured outputs Design and implement: Chain-of-thought reasoning for financial analysis Dynamic prompt routing based on query complexity Memory management for long-running financial conversations Tool integration for agents to access GL, bank feeds, and operational data Optimise token usage and response latency for real-time WhatsApp interactions API Development & Integration (FastAPI Focus) Build high-performance FastA...

🌟 We're Hiring: PySpark Data Engineer! 🌟 We are seeking an experienced PySpark Data Engineer to design, develop, and maintain large-scale data processing systems. The ideal candidate will have expertise in Apache Spark, Python, and big data technologies to build robust data pipelines and analytics solutions. 📍 Location: Hyderabad, India ⏰ Work Mode: Flexible office & remote 💼 Role: PySpark Data Engineer What You'll Do 🎯 Design and develop scalable data pipelines using PySpark 📊 Optimize data processing workflows for performance and reliability 🔧 Build ETL processes for large datasets across multiple sources ☁️ Deploy and maintain data solutions on cloud platforms 📈 Collaborate with data scie...

Design, develop, and deploy ML models for Agentic AI use cases. Work with AWS AI/ML ecosystem (SageMaker, Bedrock, Lambda, Step Functions, S3, DynamoDB, Kinesis). Preprocess and engineer features from structured, unstructured, and streaming data. Collaborate with data engineers to ensure high-quality, well-curated training datasets. Implement LLM fine-tuning, embeddings, and retrieval-augmented generation (RAG) pipelines. Evaluate and optimize models for accuracy, performance, scalability, and cost-efficiency. Integrate models into production applications and APIs. Work with MLOps teams to automate training, testing, deployment, and monitoring workflows. Perform experimentation, A/B testing,...

Role Description Key Responsibilities Design, develop, and optimize ETL pipelines using PySpark on Google Cloud Platform (GCP). Work with BigQuery, Cloud Dataflow, Cloud Composer (Apache Airflow), and Cloud Storage for data transformation and orchestration. Develop and optimize Spark-based ETL processes for large-scale data processing. Implement best practices for data governance, security, and monitoring in a cloud environment. Collaborate with data engineers, analysts, and business stakeholders to understand data requirements. Troubleshoot performance bottlenecks and optimize Spark jobs for efficient execution. Automate data workflows using Apache Airflow or Cloud Composer. Ensure data qua...

Role Description Job Title: Data Engineer – AWS & PySpark Location: Trivandrum/Kochi/Chennai/Hyderabad/Bangalore/Pune/Noida Experience Required: 5–8 years Employment Type: Full-Time Department: Data Engineering / Technology Job Summary: We are seeking a skilled Data Engineer with 5–8 years of experience to design, implement, and maintain robust, scalable data architectures on AWS . The ideal candidate will be highly proficient in PySpark and experienced with AWS cloud data services including S3, Glue, and Redshift . You will work closely with cross-functional teams to enable seamless data flows and ensure efficient ETL pipelines across our cloud infrastructure. Key Responsibilities: Design, ...

Role Description Job Title: Data Engineer – AWS & PySpark Location: Trivandrum/Kochi/Chennai/Hyderabad/Bangalore/Pune/Noida Experience Required: 5–8 years Employment Type: Full-Time Department: Data Engineering / Technology Job Summary We are seeking a skilled Data Engineer with 5–8 years of experience to design, implement, and maintain robust, scalable data architectures on AWS . The ideal candidate will be highly proficient in PySpark and experienced with AWS cloud data services including S3, Glue, and Redshift . You will work closely with cross-functional teams to enable seamless data flows and ensure efficient ETL pipelines across our cloud infrastructure. Key Responsibilities Design, im...

Job Description We are looking for a seasoned Senior Data Engineer with a passion for building scalable, reliable, and cutting-edge data solutions on cloud platforms. The ideal candidate will bring deep expertise in Google Cloud Platform (GCP), BigQuery, and modern data engineering practices, including experience with ingestion and transformation tools, as well as proficiency in medallion architecture to drive data quality and governance. Experience with Databricks on AWS is highly valued and considered a distinct advantage. As a key member of our data engineering team, you will architect and implement end-to-end data pipelines that power actionable business intelligence, advanced analytics,...

Role Description Role Proficiency: This role requires proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources. The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong coding skills in Python PySpark and SQL. This position demands independence and proficiency across various data domains. Expertise in data warehousing solutions such as Snowflake BigQuery Lakehouse and Delta Lake is essential including the ability to calculate processing costs and address performance issues. A solid understanding of DevOps and infrastructure needs is also required. Outcomes...

Role Description Role Proficiency: This role requires proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources. The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong coding skills in Python PySpark and SQL. This position demands independence and proficiency across various data domains. Expertise in data warehousing solutions such as Snowflake BigQuery Lakehouse and Delta Lake is essential including the ability to calculate processing costs and address performance issues. A solid understanding of DevOps and infrastructure needs is also required. Outcomes...

Role Description Job Title: Jr. ML Engineer Experience Range: 8+ years Must Have Skills Programming: Strong expertise in Python or R Applied Machine Learning: Problem framing – ability to choose between Supervised, Self-supervised, or Reinforcement Learning (RL) Data wrangling – experience with Weak/Distant Supervision, Pseudo-labelling, strong EDA, data preparation, labelling, and data augmentation End-to-end modelling in ML, DL, and RL Experience with Single models, Ensembles, Mixture of Experts Understanding of Mathematical Induction, Tree Induction, Deep Learning fundamentals, and optimization algorithms like SGD Transfer learning – N-shot learning (or variants), fine-tuning skills ML/DL...

Role Description Experience Range: 6-12 years Location: Trivandrum, Kochi, Bangalore Mandatory Skills Programming Languages: Scala, Spark, PySpark, Python, SQL Big Data Technologies: Hadoop, Hive, Pig, MapReduce ETL & Data Engineering: Data Warehouse Design, ETL, Data Analytics, Data Mining, Data Cleansing Cloud Platforms: GCP, Azure Tools & Frameworks: Apache Hadoop, Airflow, Kubernetes, Containers Other Skills: Data pipeline creation, optimization, troubleshooting, and data validation Work Experience: 6+ years in Data Warehouse and Big Data technologies 4+ years of hands-on experience with Scala, Spark, PySpark, Python, and SQL 3+ years in strategic data planning, governance, and standard ...

Key Responsibilities End-to-End Product Ownership: Lead the development of predictive models from exploration and prototype to full-scale production deployment. Prediction: Build robust regression and/or time-series and/or deep learning models to predict prices/values of financial assets, oil, apparel, and other commodities. Model Optimization: Continuously monitor and fine-tune models for accuracy, performance, and scalability using real-time data feedback. ML Ops & Deployment: Collaborate with engineering to ensure successful deployment and monitoring of models in production environments. Stakeholder Collaboration: Translate business problems into analytical frameworks, working closely wit...

Experience: 2 to 3 Years Employment Type: Full-time Shift: US Timings Location : T.Nagar, Chennai. Work Mode: On-site / 5 days a week. Job Description: We are seeking a motivated DevOps Engineer with hands-on experience in ETL tools , DevOps practices , SQL , Python , and AWS . The ideal candidate will be responsible for automating workflows, supporting data pipelines, and ensuring system reliability in a fast-paced environment. Key Responsibilities: Develop and manage CI/CD pipelines and infrastructure automation. Work with ETL tools to build, schedule, and monitor data workflows. Write and optimize SQL queries for data extraction and reporting. Develop Python scripts for automation and int...

Experience: 5-10years Location: Hyderabad Google Cloud + Ingestion: Experience on BigQuery, Cloud Storage or equivalent cloud platforms Knowledge of BigQuery ingress and egress patterns Experience in writing Airflow DAGs Knowledge of pubsub,dataflow or any declarative data pipeline tools using batch and streaming ingestion Other GCP Services: Vertex AI, Model Registry, Secret Manager, KMS, Composer, KubeFlow, Container Registry, Artefact Registry, Cloud Build, Cloud Run, OAuth2.0, Scheduler, GKE, Model Registry, MIG, Cloud Function, Pub/Sub Extensive experience in Google Cloud Platform (GCP) and related services (e.g. IAM, BigQuery, cloud storage, functions, compute). Creating data models an...

About Company: Our Client Corporation provides digital engineering and technology services to Forbes Global 2000 companies worldwide. Our Engineering First approach ensures we can execute all ideas and creatively solve pressing business challenges. With industry expertise and empowered agile teams, we prioritize execution early in the process for impactful results. We combine logic, creativity and curiosity to build, solve, and create. Every day, we help clients engage with new technology paradigms, creatively building solutions that solve their most pressing business challenges and move them to the forefront of their industry. Job Title : GCP Big Data Engineer Key Skills : PySpark , Airflow...

About Client: Our Client is a global IT services company headquartered in Southborough, Massachusetts, USA. Founded in 1996, with a revenue of $1.8B, with 35,000+ associates worldwide, specializes in digital engineering, and IT services company helping clients modernize their technology infrastructure, adopt cloud and AI solutions, and accelerate innovation. It partners with major firms in banking, healthcare, telecom, and media. Our Client is known for combining deep industry expertise with agile development practices, enabling scalable and cost-effective digital transformation. The company operates in over 50 locations across more than 25 countries, has delivery centers in Asia, Europe, an...

Key deliverables: Enhance and maintain the MDM platform to support business needs Develop data pipelines using Snowflake, Python, SQL, and orchestration tools like Airflow Monitor and improve system performance and troubleshoot data pipeline issues Resolve production issues and ensure platform reliability Role responsibilities: Collaborate with data engineering and analytics teams for scalable solutions Apply DevOps practices to streamline deployment and automation Integrate cloud-native tools and services (AWS, Azure) with the data platform Utilize dbt and version control (Git) for data transformation and management