3177 Data Pipelines Jobs - Page 18

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : AWS Glue, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and im...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Python on Azure Good to have skills : NA Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Custom Software Engineer, you will develop custom software solutions to design, code, and enhance components across systems or applications. Your typical day will involve collaborating with cross-functional team...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : AWS Glue, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and im...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : AWS Glue, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and im...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : AWS Glue, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and im...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : AWS Glue, PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and im...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : Oracle Procedural Language Extensions to SQL (PLSQL), Data Engineering, Google Cloud Platform Architecture Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Python (Programming Language), Microsoft SQL Server, Databricks Unified Data Analytics Platform, PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processin...

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : AI Agents & Workflow Integration Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a AI agentic Engineer, you will define the data requirements and structure for the application Embedded FDE engineer (5–10 yrs) building AI-first SAP cloud applications and end-to-end solutions ...

About The Role Project Role : Custom Software Engineer Project Role Description : Analyze, design, code and test multiple components of application code across one or more clients. Perform maintenance, enhancements and/or development work. Must have skills : PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to effectively migrate and deploy data across various ...

We are looking for a skilled professional with 6-17 years of experience to join our team as an Agentic AI expert. The ideal candidate will have a strong background in Artificial Intelligence and experience working with agent frameworks. Roles and Responsibility Design and develop innovative AI solutions using various agent frameworks. Collaborate with cross-functional teams to integrate AI models into existing systems. Develop and implement automated testing scripts to ensure high-quality AI solutions. Troubleshoot and resolve complex technical issues related to AI model integration. Work closely with data engineers to design and implement data pipelines for AI model training. Develop and ma...

We are looking for a skilled Big Data Developer with 6-8 years of experience to join our team in Hyderabad/Pune. The ideal candidate will have strong knowledge of Unix/BigData Scripting and a solid understanding of BigData (CDP/Hive) Environment. Roles and Responsibility Design, develop, and implement big data solutions using Hadoop/CDP, Hive. Collaborate with the Lead Hadoop Data Engineer and Product Owner to achieve project goals. Support existing applications and design/build new Data Pipelines. Participate in service management and issue resolution activities. Ensure efficient use of Hive, Unix Scripting, and Control-M to reduce lead time to delivery. Provide solutions and tools to enhan...

We are looking for a skilled Snowflake Developer with 5-20 years of experience to join our team in Chennai. The ideal candidate will have strong expertise in Snowflake, ETL, and data ingestion pipelines. Roles and Responsibility Design, develop, and implement data pipelines using Snowflake. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain complex SQL queries and PL/SQL scripts. Work closely with stakeholders to understand business needs and provide technical solutions. Ensure high-quality deliverables and meet project deadlines. Participate in code reviews and contribute to the improvement of the overall codebase. Job Requirements ...

Looking to onboard a skilled professional with 8-16 years of experience in the field, located anywhere in India. Roles and Responsibility Design and develop scalable data pipelines using Google Data Products such as Cloud Data Fusion, BigQuery, Dataflow, Dataproc, AI Building Blocks, Looker, Dataprep, etc. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data engineering solutions using E2E Data Engineering and Lifecycle management techniques. Ensure data quality and metadata management through effective MDM tools. Implement data visualization literacy for business insights. Participate in prototyping, usability testing...

Primary skills: Hana, Hadoop, Redis, Kafka, InfluxDB, Postgres, Cassandra, and MySQL Terraform and Ansible Python Primary Responsibilities The Cloud Database Engineer provides 24x7 operational support and designs and creates automation for the Hana, Hadoop, Redis, Kafka, InfluxDB, Postgres, Cassandra, and MySQL instances supporting the SAP Procurement and Business Network cloud platform in public cloud, SAP Business Technology Platform, and SAP Managed Data Centers Data Dynamo: Design, deploy, and maintain highly available, scalable, and secure cloud databases to support real-time data processing and fuel data-driven insights. Support Superhero: Provide 24x7 support, monitoring, and proactiv...

We are looking for a skilled Data Scientist GenAI Developer with 4 to 10 years of experience. The ideal candidate will have expertise in developing Generative AI-based applications and knowledge of various GenAI techniques and frameworks. This position is available in Mumbai, Pune, and Hyderabad. Roles and Responsibility Develop and deploy Machine Learning models using Python. Design and implement API development for data exchange between systems. Collaborate with cross-functional teams to integrate AI solutions into existing systems. Work on Azure AI services to develop and deploy scalable AI solutions. Develop and maintain large-scale data pipelines and architectures. Ensure high-quality d...

We are looking for a skilled Python developer with expertise in Airflow to join our team at IDESLABS PRIVATE LIMITED. The ideal candidate should have 6-10 years of experience. Roles and Responsibility Design, develop, and deploy scalable data pipelines using Apache Airflow. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop complex data transformations and scripts using Python. Troubleshoot and optimize existing data pipelines for improved efficiency. Ensure data quality and integrity by implementing robust testing and validation procedures. Participate in code reviews and contribute to the improvement of the overall codebase. Job Requirements St...

We are looking for a skilled Python developer with expertise in Airflow to join our team at IDESLABS PRIVATE LIMITED. The ideal candidate should have 6-10 years of experience. Roles and Responsibility Design, develop, and deploy scalable data pipelines using Apache Airflow. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop complex data transformations and scripts using Python. Troubleshoot and optimize existing data pipelines for improved efficiency. Ensure data quality and integrity by implementing robust testing and validation procedures. Participate in code reviews and contribute to the improvement of the overall codebase. Job Requirements St...

We are looking for a skilled professional with 5-8 years of experience in developing AI and Machine learning workloads, especially in the field of Natural Language Processing (NLP). The ideal candidate will have expertise in Python programming for Data science workloads and automation and production deployment of NLP and GenAI models using PowerApps. This position is located in Bangalore. Roles and Responsibility Design and build high-performing AI agents and virtual chatbots using MS Copilot Studio. Develop Generative AI solutions, virtual assistants, and chatbots using Azure Open AI or similar LLMs. Build and deploy NLP and GenAI models using PowerApps. Collaborate with cross-functional te...

We are looking for a skilled Data Engineer with expertise in Apache Spark and Hive to join our team at IDESLABS PRIVATE LIMITED. The ideal candidate will have 4-6 years of experience. Roles and Responsibility Design, develop, and implement data pipelines using Apache Spark and Hive. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data processing systems using Python, Scala, or Java. Work with Kafka, Solace, and object stores such as S3 and MinIO. Ensure data quality and integrity by implementing robust testing and validation frameworks. Participate in code reviews and contribute to improving overall code quality. Job R...

Responsibilities: designing, developing, and maintaining scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Live Tables. Collaborate with cross-functional teams to understand data requirements and translate them into efficient data models and pipelines. Implement best practices for data engineering, including data quality, and data security. Optimize and troubleshoot complex data workflows to ensure high performance and reliability. Develop and maintain documentation for data engineering processes and solutions. Requirements: Bachelor's or Master's degree. Proven experience as a Data Engineer, with a focus on Databricks, PySpark, Spark SQL, and Delta Live Tables. Strong ...

We are looking for a highly skilled Data Engineer with expertise in Snowflake and ETL to join our team at IDESLABS PRIVATE LIMITED. The ideal candidate will have a strong background in data engineering and experience working with large datasets. Roles and Responsibility Design, develop, and implement data pipelines using Snowflake and ETL tools. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain databases and data systems to support business intelligence and analytics. Ensure data quality and integrity by implementing data validation and testing procedures. Optimize data processing workflows to improve performance and efficiency. Tro...

Senior Python Developer (6 Years Experience) Location : Pune/Bangalore Employment Type: Full-time/Contract Experience : 6 Years Role Overview: We are seeking a Senior Python Developer with 6 years of experience, specializing in database technologies like DB2 or Snowflake. The ideal candidate will have a strong background in backend development, data processing, and performance optimization. Experience with Java is a plus. Key Responsibilities: Design, develop, and maintain scalable and efficient Python applications Work extensively with DB2 or Snowflake for data modeling, query optimization, and performance tuning Develop and optimize SQL queries, stored procedures, and data pipelines Collab...

Objective: Build and maintain data pipelines and infrastructure to support data storage and processing. Key Responsibilities : - Design, develop, and optimize data pipelines using Azure Databricks. - Ensure data reliability, scalability, and quality. - Integrate data sources into a unified platform. - Monitor and troubleshoot data workflows. Skills Required : - Expertise in Azure Databricks, Spark, and big data tools. - Strong programming skills (e.g., Python, Scala, SQL). - Knowledge of ETL processes and cloud technologies. - Familiarity with CI/CD practices for data pipelines. - -Developer / Software Engineer - Four to Six Years,Azure Databricks - Four to Six Years-PSP Defined SCU in Data ...

PRIMARY SKILL - BigQuery, DBT, Python, Production Support Design, develop, and maintain scalable data pipelines using BigQuery, DBT, and PythonImplement and optimize ETL/ELT workflows on Google Cloud Platform (GCP) using services like Cloud Composer, Dataflow, and Cloud Storage Provide production support for data workflows, ensuring high availability, performance, and data integrityMonitor and troubleshoot issues in batch and streaming pipelines Collaborate with cross-functional teams to gather requirements and deliver robust data solutionsAutomate operational tasks and build alerting mechanisms using Python scripts and GCP monitoring tools Maintain and enhance DBT models, ensuring modularit...