1159 Data Pipeline Jobs

Data Engineer Experience - 7 to 10 Yrs Location - Bangalore Role Summary: The Data Engineer is the foundation of the AI CoE. You build and maintain the robust, scalable, and efficient data pipelines that feed clean, reliable data to the data scientists and ML models. You ensure that data is accessible, organized, and of high quality. Key Responsibilities: Design, construct, install, and maintain large-scale data processing systems and data lakes/warehouses. Build scalable and reliable ETL/ELT pipelines to ingest data from diverse sources. Ensure data quality and implement data governance and security standards. Collaborate with Data Scientists to understand data requirements for modeling. Op...

REConnect Energy's GRIDConnect platform helps integrate and manage energy generation and consumption for 1000s of renewable energy assets and grid operators. We are currently serving customers across India, Bhutan and the Middle East with expansion planned in US and European markets. We are headquartered in Central Bangalore with a team of 150+ and growing. You will jointhe Bangalore based Engineering team as a senior member and work at the intersection of Energy, Weather & Climate Sciences and AI. Responsibilities: Engineering - Take complete ownership of engineering stacks including Data Engineering and MLOps. Define and maintain software systems architecture for high availability 24x7 sys...

Job Title : AI Solution Engineer Location : Chennai Employment Type : Full-time Department : Product Engineering. About Us: Newage Software (www.newage-global.com) is a leading provider of freight forwarding software and digitized back-office services designed to meet the evolving needs of the freight forwarding and logistics industry. Our end-to-end SaaS-based ERP solutions and digital back-office services enhance operational efficiency, reduce cost leakages, and provide actionable insights for our global customers. Built on advanced technologies and automation, our solutions cater to businesses of all sizes : small, medium, and large supporting functions across sales, finance, accounting, ...

Need - candidate who having degree in data scientist either in graduation or post graduation Key Responsibilities: Analyze large datasets to extract meaningful insights and patterns to support business decisions. Develop, implement, and optimize machine learning and statistical models for predictive and prescriptive analytics. Collaborate with engineering, product, and business teams to define and deliver data-driven solutions. Design and build data pipelines and perform data wrangling using Python, R, or SQL. Communicate analytical findings through clear data visualizations and storytelling techniques. Evaluate model performance and improve accuracy and efficiency. Work with cloud platforms...

Duties & Responsibilities Data processing: data querying, analysis and validation. Build, optimize and maintain data flows within data warehouses, storage and cloud environments. Data Modeling: Design scalable data pipeline schemas and architecture. Deploy and monitor data pipelines in production. Basic Qualifications Bachelors degree in computer science, Information Systems, Engineering, or a related field • Proficient in Python, SQL, and Spark/PySpark. • Design, develop, and maintain efficient and scalable data pipelines for both batch and real-time processing. • Build and manage data integration workflows between internal and external data sources. • Implement and optimize data models, sc...

About The Opportunity : A fast-scaling technology company operating in the Enterprise AI / Generative AI sector, building production-grade LLM-driven products and intelligent automation for global clients. We deliver secure, low-latency generative systems that power conversational AI, summarization, code generation, and retrieval-augmented applications across cloud-native environments. We are hiring a Senior Generative AI Engineer (7+ years) to own architecture, model development, and production deployment of advanced generative systems. This is a fully remote role for candidates based in India. Role & Responsibilities : - Lead end-to-end design and delivery of Generative AI/LLM solutionsdat...

About The Opportunity : A fast-scaling technology company operating in the Enterprise AI / Generative AI sector, building production-grade LLM-driven products and intelligent automation for global clients. We deliver secure, low-latency generative systems that power conversational AI, summarization, code generation, and retrieval-augmented applications across cloud-native environments. We are hiring a Senior Generative AI Engineer (7+ years) to own architecture, model development, and production deployment of advanced generative systems. This is a fully remote role for candidates based in India. Role & Responsibilities : - Lead end-to-end design and delivery of Generative AI/LLM solutionsdat...

About The Opportunity : We are seeking a highly skilled Lead Data Engineer with strong expertise in Python, Pandas, PySpark, AWS, and SQL to design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high data quality and performance across cloud-based environments. Role & Responsibilities : - Lead the design and development of data pipelines for ingestion, transformation, and integration from multiple sources into the enterprise data platform. - Implement data quality frameworks, validation checks, and monitoring solutions using Python and ...

Job description: We are seeking a highly skilled and experienced Senior Data Warehouse Developer to design, develop, and optimize our enterprise data warehouse solutions. This role requires strong technical expertise in data modeling, ETL and semantic layer development, basic SQL performance tuning with a focus on building scalable, secure, and high-performing data platforms. The ideal candidate will combine deep expertise in data modeling, architecture, and integration with the ability to collaborate across business and technology teams. Please apply, If you have the passion and standard methodologies work in an environment where challenges are a norm, where individual brilliance is valued ...

Key Responsibilities : - Develop, implement, and maintain complex quantitative models and algorithms for high-frequency trading, risk management, or portfolio optimization. - Write high-performance, low-latency code in languages like C++ or Java to ensure real-time system efficiency. - Collaborate closely with quantitative researchers, traders, and other stakeholders to translate mathematical models into production-level code. - Design and build data pipelines for efficient data acquisition, cleaning, and processing from various financial market sources. - Conduct thorough backtesting and validation of models to ensure accuracy and reliability under various market conditions. - Optimize and ...

Role & Responsibilities : - Design, develop, and maintain robust AWS-based data pipelines using Python and PySpark. - Implement efficient ETL processes, ensuring data integrity and optimal performance across AWS services (S3, Glue, EMR, Redshift). - Collaborate with cross-functional teams to integrate data engineering solutions within broader business-critical applications. - Troubleshoot and optimize existing data workflows, ensuring high availability, scalability, and security of cloud solutions. - Exercise best practices in coding, version control, and documentation to maintain a high standard of engineering excellence. Skills & Qualifications : Must-Have : 4+ years of hands-on experience...

At Meazure Learning , we aim to empower open-minded, inquisitive, and driven people, and we love how each new addition to the team adds to our culture. Here, you can positively impact the careers of millions of test-takers and reshape the education technology landscape by working toward one vision: To provide life-long learners across the globe the freedom to pursue their academic dreams and career aspirations safely, securely, and conveniently.” Our Vision starts with our values— Hungry, Humble, and Smart. We are looking for team members who model drive for results, think of the organization and team first without ego or agenda, and lead with emotional intelligence. This is the foundation o...

Job role: AWS Data Engineer Location: PAN India Job description Key Responsibilities Design and build data pipelines & Data lakes to automate ingestion of structured and unstructured data that provide fast, optimized, and robust end-to-end solutions Knowledge about the concepts of data lake and dat warehouse Experience working with AWS big data technologies Improve the data quality and reliability of data pipelines through monitoring, validation and failure detection. Deploy and configure components to production environments Technology: Redshift, S3, AWS Glue, Lambda, SQL, PySpark, SQL.

Job role: AWS Data Engineer Location: PAN India Job description Key Responsibilities Design and build data pipelines & Data lakes to automate ingestion of structured and unstructured data that provide fast, optimized, and robust end-to-end solutions Knowledge about the concepts of data lake and dat warehouse Experience working with AWS big data technologies Improve the data quality and reliability of data pipelines through monitoring, validation and failure detection. Deploy and configure components to production environments Technology: Redshift, S3, AWS Glue, Lambda, SQL, PySpark, SQL.

TCS Hiring for skill " ML developer ". Role: ML developer Experience: 2-6 years Work Location: PAN India Responsibility of / Expectations from the Role: 1. Data Pipeline Management 2. Production Deployment and Monitoring 3. Performance Optimization and Testing 4. Proficiency in Python 5. Machine Learning Frameworks 6. Applied Mathematics and Statistics 7. Software Engineering Principles Note: Virtual/Video call interview will be on 15th Nov 2025 @10.00 AM to 5.00 PM (1-2pm lunch break). Note: Applications will be shortlisted based upon response received & Eligibility criteria.

We are looking for a highly skilled Machine Learning Engineer III to join our team at Yum Brands. The ideal candidate will have a strong background in machine learning and software development, with excellent problem-solving skills and the ability to work collaboratively in a fast-paced environment. Roles and Responsibility Design, develop, and deploy machine learning models using various algorithms and techniques. Collaborate with cross-functional teams to identify business problems and develop solutions. Develop and maintain large-scale data pipelines and architectures. Conduct research and stay updated on new machine learning technologies and trends. Work closely with stakeholders to unde...

We are looking for a highly skilled and experienced Cloud Data Engineer to join our team at Apple India Pvt Ltd. The ideal candidate will have a strong background in designing, developing, and implementing data engineering solutions on cloud platforms. Roles and Responsibility Design and develop scalable and efficient data pipelines using cloud-based technologies. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data architectures on cloud platforms such as AWS or Azure. Ensure data quality and integrity by implementing data validation and testing procedures. Optimize data processing workflows to improve performance and...

A Machine Learning Engineer is responsible for designing, building, and maintaining systems that use artificial intelligence (AI) and machine learning (ML) algorithms to solve complex problems. 1. Developing and implementing ML algorithms: The Machine Learning Engineer develops and implements machine learning algorithms to solve specific problems, such as natural language processing, computer vision, or predictive modeling. 2. Building data pipelines: The Machine Learning Engineer is responsible for building data pipelines that collect, store, and preprocess data used in machine learning algorithms. 3. Creating and maintaining ML infrastructure: The Machine Learning Engineer is responsible f...

We are looking for a highly skilled Machine Learning Engineer with 2 to 7 years of experience to join our team at Apple India Pvt Ltd. The ideal candidate will have expertise in machine learning and natural language processing. Roles and Responsibility Design, develop, and deploy machine learning models using various algorithms and techniques. Collaborate with cross-functional teams to integrate machine learning solutions into existing systems. Develop and maintain large-scale data pipelines and architectures. Conduct research and experimentation to improve model performance and efficiency. Work closely with data engineers to ensure seamless data flow and processing. Implement automated test...

Job Title: Senior Data Engineer (Kafka & AWS) Responsibilities: Develop and maintain real-time data pipelines using Apache Kafka (MSK or Confluent) and AWS services . Configure and manage Kafka connectors , ensuring seamless data flow and integration across systems. Demonstrate strong expertise in the Kafka ecosystem , including producers, consumers, brokers, topics, and schema registry. Design and implement scalable ETL/ELT workflows to efficiently process large volumes of data. Optimize data lake and data warehouse solutions using AWS services such as Lambda, S3, and Glue . Implement robust monitoring, testing, and observability practices to ensure reliability and performance of data platf...

Key Responsibilities Design, develop, and optimize ETL pipelines using PySpark on Google Cloud Platform (GCP) . Work with BigQuery , Cloud Dataflow , Cloud Composer (Apache Airflow) , and Cloud Storage for data transformation and orchestration. Develop and optimize Spark-based ETL processes for large-scale data processing . Implement best practices for data governance, security, and monitoring in a cloud environment. Collaborate with data engineers, analysts, and business stakeholders to understand data requirements. Troubleshoot performance bottlenecks and optimize Spark jobs for efficient execution. Automate data workflows using Apache Airflow or Cloud Composer . Ensure data quality, valid...

Role: Sr AWS Data engineer - Data Migration Experience: 6+ years Location: Hyderabad Fission Labs , headquartered in Sunnyvale, with offices in Dallas & Hyderabad, Fission Labs is a leading software development company, specializing in crafting flexible, agile, and scalable solutions that propel businesses forward. With a comprehensive range of services, including product development, cloud engineering, big data analytics, QA, DevOps consulting, and AI/ML solutions, we empower clients to achieve sustainable digital transformation that aligns seamlessly with their business goals. Responsibilities Experience leading and driving large scale data migration projects Experience in working with all...

Job Description Job Summary: We are looking for a proactive and technically strong Lead Gen AI Engineer 10+ years exp to lead a high-performing offshore team working on Gen AI solutions in the real estate domain. This role requires close collaboration with the Gen AI Architect and direct accountability for delivery, technical execution, and mentoring the team across various Gen AI initiatives. Key Responsibilities: - Lead the offshore team in developing, testing, and deploying Gen AI and NLP-based applications. - Collaborate closely work with the Lead Architect to translate high-level designs into actionable workstreams. - Develop and guide the team on best practices in GPT-based development...

Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be skilled in ETL tools such as Informatica Glue Databricks and DataProc with coding expertise in Python PySpark and SQL. Works independently and has a deep understanding of data warehousing solutions including Snowflake BigQuery Lakehouse and Delta Lake. Capable of calculating costs and understanding performance issues related to data solutions. Outcomes: Act creatively to develop pipelines and applications by selecting appropriate technical options optimizing application development mainten...

Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc with coding skills in Python PySpark and SQL. Works independently and demonstrates proficiency in at least one domain related to data with a solid understanding of SCD concepts and data warehousing principles. Outcomes: Collaborate closely with data analysts data scientists and other stakeholders to ensure data accessibility quality and security across various data sources.rnDesign develop and maintain data pipelines...