757 Data Pipelines Jobs - Page 29

IBM Consulting Overview A career in IBM Consulting is rooted by long-term relationships and close collaboration with clients across the globe. You'll work with visionaries across multiple industries to improve the hybrid cloud and AI journey for the most innovative and valuable companies in the world. Your ability to accelerate impact and make meaningful change for your clients is enabled by our strategic partner ecosystem and our robust technology platforms across the IBM portfolio, including Software and Red Hat. Curiosity and a constant quest for knowledge serve as the foundation to success in IBM Consulting. In your role, you'll be encouraged to challenge the norm, investigate ideas outs...

In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and ad...

We are looking for a highly experienced Lead Data Engineer / Data Architect to lead the design, development, and implementation of scalable data pipelines, data Lakehouse, and data warehousing solutions. The ideal candidate will provide technical leadership to a team of data engineers, drive architectural decisions, and ensure best practices in data engineering. This role is critical in enabling data-driven decision-making and modernizing our data infrastructure. Key Responsibilities: Act as a technical leader responsible for guiding the design, development, and implementation of data pipelines, data Lakehouse, and data warehousing solutions. Lead a team of data engineers, ensuring adherence...

We are hiring a Senior Data Engineer for a 6-month remote contract position. The ideal candidate is highly skilled in building scalable data pipelines and working within the Azure cloud ecosystem, especially Databricks, ADF, and PySpark. You'll work closely with cross-functional teams to deliver enterprise-level data engineering solutions. #KeyResponsibilities Build scalable ETL pipelines and implement robust data solutions in Azure. Manage and orchestrate workflows using ADF, Databricks, ADLS Gen2, and Key Vaults. Design and maintain secure and efficient data lake architecture. Work with stakeholders to gather data requirements and translate them into technical specs. Implement CI/CD pipeli...

Roles & Responsibilities Lead and manage the enterprise data operations team responsible for data ingestion, processing, validation, quality control, and publishing Define and implement SOPs for data lifecycle management ensuring data accuracy, completeness, and integrity Oversee and improve daily operational workflows: scheduling, monitoring, troubleshooting data jobs across cloud and on-prem environments Establish and track key data operations metrics (SLAs, throughput, latency, data quality, incident resolution) and drive continuous improvement Partner with data engineering and platform teams to optimize pipelines, support integrations, and ensure scalability and resilience Collaborate wi...

Req ID: 323226 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Digital Solution Architect Sr. Advisor to join our team in Bengaluru, India, Karn?taka (IN-KA), India (IN). Key Responsibilities: Design data platform architectures (data lakes, lakehouses, DWH) using modern cloud-native tools (e.g., Databricks, Snowflake, BigQuery, Synapse, Redshift). Architect data ingestion, transformation, and consumption pipelines using batch and streaming methods. Enable real-time analytics and machine learning through scalable and ...

Introduction In this role, youll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology In this role, youll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovat...

Introduction In this role, youll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. In this role, youll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innova...

Job Summary: We are seeking an experienced Snowflake Data Engineer to design, develop, and optimize data pipelines and data architecture using the Snowflake cloud data platform. The ideal candidate will have a strong background in data warehousing, ETL/ELT processes, and cloud platforms, with a focus on creating scalable and high-performance solutions for data integration and analytics. --- Key Responsibilities: * Design and implement data ingestion, transformation, and loading processes (ETL/ELT) using Snowflake. * Build and maintain scalable data pipelines using tools such as dbt, Apache Airflow, or similar orchestration tools. * Optimize data storage and query performance in Snowflake usi...

JobOpening Senior Data Engineer (Remote, Contract 6 Months) Remote | Contract Duration: 6 Months | Experience: 6-8 Years We are hiring a Senior Data Engineer for a 6-month remote contract position. The ideal candidate is highly skilled in building scalable data pipelines and working within the Azure cloud ecosystem, especially Databricks, ADF, and PySpark. You'll work closely with cross-functional teams to deliver enterprise-level data engineering solutions. #KeyResponsibilities Build scalable ETL pipelines and implement robust data solutions in Azure. Manage and orchestrate workflows using ADF, Databricks, ADLS Gen2, and Key Vaults. Design and maintain secure and efficient data lake archite...

JobOpening Senior Data Engineer (Remote, Contract 6 Months) Remote | Contract Duration: 6 Months | Experience: 6-8 Years We are hiring a Senior Data Engineer for a 6-month remote contract position. The ideal candidate is highly skilled in building scalable data pipelines and working within the Azure cloud ecosystem, especially Databricks, ADF, and PySpark. You'll work closely with cross-functional teams to deliver enterprise-level data engineering solutions. #KeyResponsibilities Build scalable ETL pipelines and implement robust data solutions in Azure. Manage and orchestrate workflows using ADF, Databricks, ADLS Gen2, and Key Vaults. Design and maintain secure and efficient data lake archite...

We are looking for a highly skilled and experienced Senior Associate, Enterprise Integration Engineer to join our team in Bengaluru. The ideal candidate will have 2-7 years of experience in data migration, application development, or system integration. Roles and Responsibility Support the design and implementation of data migration and system integration projects. Collaborate with ERP, CRM, and HCM teams to gather and review business and technical requirements. Develop, test, and deploy integrations and data pipelines using modern integration platforms. Conduct unit testing and assist with QA to ensure technical solutions meet client needs and follow best practices. Take ownership of indivi...

Build and deploy scalable ML models and MLOps pipelines in collaboration with data scientists Required Candidate profile 6–12 yrs in ML development, Python, model tuning, and enterprise AI deployment.

Python, software development & application & training of AI models, data pipelines, data structures & data integration LLM-based AI frameworks OpenAI CI/CD,serverlessAWS platforms, containerization, data structures, data platforms & data integration

Snowflake database administration designing data architecture, database networking, maintaining data security, configuring user access Python, git, SQL, AWS Snowflake data pipelines, data structures, data platforms, data integration, data governance

We are looking for a Senior Data Engineer who will design, build, and maintain scalable data pipelines and ingestion frameworks. The ideal candidate must have experience with DBT, orchestration tools like Airflow or Prefect, and cloud platforms such as AWS. Responsibilities include developing ELT pipelines, optimizing queries, implementing CI/CD, and integrating with AWS services. Strong SQL, Python, and data modeling skills are essential. The role also involves working with real-time and batch processing, ensuring high performance and data integrity.

We are looking for an experienced Software Developer with 5+ years of expertise in Java or Python for data processing and automation. The ideal candidate should have strong proficiency in Java, Spring, Microservices, and REST API development. Experience with cloud platforms, specifically Google Cloud Platform (GCP), is required. The candidate must have hands-on experience in creating IDE plugins for PyCharm, VS Code, IntelliJ, and web consoles. Additionally, proficiency in Python and data pipeline development is mandatory. The role demands a deep understanding of software development principles, automation, and scalable system design.

We are seeking a skilled Big Data Developer with 3+ years of experience to develop, maintain, and optimize large-scale data pipelines using frameworks like Spark, PySpark, and Airflow. The role involves working with SQL, Impala, Hive, and PL/SQL for advanced data transformations and analytics, designing scalable data storage systems, and integrating structured and unstructured data using tools like Sqoop. The ideal candidate will collaborate with cross-functional teams to implement data warehousing strategies and leverage BI tools for insights. Proficiency in Python programming, workflow orchestration with Airflow, and Unix/Linux environments is essential. Locations : Mumbai, Delhi / NCR, Be...

We are seeking a skilled Big Data Developer with 3+ years of experience to develop, maintain, and optimize large-scale data pipelines using frameworks like Spark, PySpark, and Airflow. The role involves working with SQL, Impala, Hive, and PL/SQL for advanced data transformations and analytics, designing scalable data storage systems, and integrating structured and unstructured data using tools like Sqoop. The ideal candidate will collaborate with cross-functional teams to implement data warehousing strategies and leverage BI tools for insights. Proficiency in Python programming, workflow orchestration with Airflow, and Unix/Linux environments is essential. Location: Remote- Delhi / NCR,Banga...

AI/ML Architecture Development Implement and deploy machine learning models in production environments Work with generative AI models and NLP techniques (e.g., OpenAI, Claude APIs) Apply prompt engineering and retrieval-augmented generation (RAG) techniques Build end-to-end ML workflows from data preprocessing to model evaluation Contribute to the development of conversational AI solutions Software Engineering Data Pipeline Development Design, build, and maintain data pipelines and end-to-end ML workflows. Build and deploy web services that integrate ML models Implement APIs using frameworks like FastAPI or Flask Work with databases (preferably MySql) and data processing tools Ensure code qu...

AI/ML Architecture Development Implement and deploy machine learning models in production environments Work with generative AI models and NLP techniques (e.g., OpenAI, Claude APIs) Apply prompt engineering and retrieval-augmented generation (RAG) techniques Build end-to-end ML workflows from data preprocessing to model evaluation Contribute to the development of conversational AI solutions Software Engineering Data Pipeline Development Design, build, and maintain data pipelines and end-to-end ML workflows. Build and deploy web services that integrate ML models Implement APIs using frameworks like FastAPI or Flask Work with databases (preferably MySql) and data processing tools Ensure code qu...

We Are the Perfect Match If You: Have at least 1 year of experience in product data management , and hold an engineering degree . Have strong business acumen and product thinking, with the ability to translate data into actionable insights to drive business growth. Possess expertise in SQL , Advanced Excel , Google Sheets , and other database tools. Have experience working with large datasets and applying data engineering best practices . Are skilled in creating, managing, and optimizing data pipelines and data warehouses . Demonstrate the ability to convert raw data into actionable reports with real-time data fetching. Can build business use cases through thorough data analysis. Bring a str...

About We are hiring a Lead Data Solutions Engineer with expertise in PySpark, Python, and preferably Palantir Foundry. You will focus on transforming complex operational data into clear customer communications for Planned Power Outages (PPO) within the energy sector. Role & responsibilities Build, enhance, and manage scalable data pipelines using PySpark and Python to process dynamic operational data. Interpret and consolidate backend system changes into single-source customer notifications. Leverage Foundry or equivalent platforms to build dynamic data models and operational views. Act as a problem owner for outage communication workflows and edge cases. Collaborate with operations and comm...

We are looking for a highly skilled and experienced Azure Data Engineer with 2 to 7 years of experience to join our team. The ideal candidate should have expertise in Azure Synapse Analytics, PySpark, Azure Data Factory, ADLS Gen2, SQL DW, T-SQL, and other relevant technologies. ### Roles and Responsibilities Design, develop, and implement data pipelines using Azure Data Factory or Azure Synapse Analytics. Develop and maintain data warehouses or data lakes using various tools and technologies. Work with various types of data sources including flat files, JSON, and databases. Build workflows and pipelines in Azure Synapse Analytics. Collaborate with cross-functional teams to identify and prio...

We are looking for a highly skilled and experienced Data Engineering Lead to join our team. The ideal candidate will have 5-10 years of experience in designing and implementing scalable data lake architecture and data pipelines. ### Roles and Responsibility Design and implement scalable data lake architectures using Azure Data Lake services. Develop and maintain data pipelines to ingest data from various sources. Optimize data storage and retrieval processes for efficiency and performance. Ensure data security and compliance with industry standards. Collaborate with data scientists and analysts to facilitate data accessibility. Monitor and troubleshoot data pipeline issues to ensure reliabil...