1617 Data Pipelines Jobs - Page 4

We are looking for a skilled AWS/Azure Cloud Engineer with 5 to 10 years of experience to join our team in Gurgaon. The ideal candidate will have prior experience with writing and debugging Python, building data pipelines, and working with cloud-based tools. Roles and Responsibility Design, develop, and implement scalable and secure cloud-based systems on AWS/Azure. Build and optimize big data pipelines using PySpark and other relevant technologies. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain technical documentation for cloud-based systems and applications. Troubleshoot and resolve complex technical issues related to cloud inf...

We are looking for a skilled SQL Database Developer with 5 to 10 years of experience to join our team in Hyderabad or Gurgaon. The ideal candidate will have expertise in developing and maintaining databases, including schema design, stored procedures, and data integrity. Roles and Responsibility Design, develop, and implement database solutions using Microsoft SQL Server and Azure SQL Database. Develop and maintain large-scale databases, ensuring high performance, security, and scalability. Collaborate with cross-functional teams to identify and prioritize database requirements. Develop and implement data pipelines using Azure Data Factory. Ensure data integrity and consistency across the or...

We are looking for a skilled SQL Database Developer with 4 to 9 years of experience to join our team in Hyderabad or Gurgaon. The ideal candidate will have a strong background in database administration and development, with expertise in Microsoft SQL Server, Azure SQL Database, and data pipelines using Azure Data Factory. Roles and Responsibility Design, develop, and maintain large-scale databases in Azure, including schema design, stored procedures, and data integrity. Continuously improve data pipelines using Azure Data Factory for efficient data processing. Develop insightful and interactive business reports using Power BI. Collaborate with cross-functional teams to identify and prioriti...

We are looking for a skilled SQL Database Developer with 4 to 8 years of experience to join our team in Hyderabad or Gurgaon. The ideal candidate will have a strong background in database administration and development, with expertise in Microsoft SQL Server and Azure SQL Database. Roles and Responsibility Design, develop, and maintain complex databases using Microsoft SQL Server and Azure SQL Database. Develop and implement data pipelines using Azure Data Factory for continuous improvement. Ensure data integrity and security by implementing best practices and standards. Collaborate with cross-functional teams to identify and prioritize project requirements. Troubleshoot and resolve database...

We are looking for highly motivated people with proven track record to conduct research in natural language processing, artificial intelligence, and machine learning. As a Research Intern, you will be working with scientists in GE Aerospace Research to develop search and recommendation systems to improve the productivity of our Engineering teams. Your responsibilities will include developing and implementing algorithms to process Aerospace domain data for recommendation systems, design experiments, conduct thorough evaluations, and document your work (e.g., publications, invention disclosures). It will also include effectively communicating your findings with the appropriate stakeholders. Yo...

The Principal Data Engineer sets the technical direction that assists engineers and data engineers to take on large, complex data problems/initiatives. This role has considerable technical influence within the Data and Software Engineering teams at JustGiving. This individual acts as an evangelist for proper data strategy with other teams at JustGiving. What youll do: Maintain and assess a legacy SQL Server-based Kimball-style data warehouse, identifying opportunities for modernization and optimization. Design and build scalable data pipelines and ETL/ELT processes using modern tools and frameworks. Collaborate with peers to define and evolve reusable tools and pipeline patterns that support...

Key Responsibilities: Optimize scalable data pipelines for ingestion, transformation, and loading of large datasets from diverse sources. Utilize Azure Data Factory for orchestration and automation of ETL/ELT processes, ensuring efficient data flow. Work extensively with Azure services such as Azure Blob Storage and Azure Data Lake Storage (ADLS Gen2) for efficient data storage and management. Integrate and manage data flows using Azure Logic Apps for event-driven automation and workflow orchestration. Develop and maintain robust SQL queries, stored procedures, and data models to support reporting and analytical requirements. Implement and enforce data governance, quality, and security best ...

The Principal Data Engineer will be a senior individual contributor responsible for designing, building, and optimizing advanced data solutions that power enterprise-wide analytics, AI/ML, and data-driven decision-making. Reporting directly to the Senior Director of Data Platforms, this role will serve as a technical expert and thought partner in scaling modern data architectures, enabling AI orchestration, and delivering robust, secure, and compliant data products. This position is highly hands-on, requiring expertise in data engineering, Databricks, Python, AI/ML platforms, and orchestration frameworks. The Principal Engineer will work across functional teams to design and implement high-p...

As a Senior Cloud Data Platform (Azure) Engineer at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team to en...

Role Description As a Senior Cloud Data Platform (Azure) Engineer at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the se...

We are seeking a skilled Lead Data Engineer with extensive experience in Snowflake, ADF, SQL, and other relevant data technologies to join our team. As a key member of our data engineering team, you will play an instrumental role in designing, developing, and managing data pipelines, working closely with cross-functional teams to drive the success of our data initiatives. Key Responsibilities : Design, implement, and maintain data solutions using Snowflake, ADF, and SQL Server to ensure data integrity, scalability, and high performance Lead and contribute to the development of data pipelines, ETL processes, and data integration solutions, ensuring the smooth extraction, transformation, and l...

Join us as a Data Engineer Youll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-making Well look to you to drive the build of effortless, digital first customer experiences If youre ready for a new challenge and want to make a far-reaching impact through your work, this could be the opportunity youre looking for We're offering this role at vice president level What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and se...

Join us as a Data Engineer Youll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-making Well look to you to drive the build of effortless, digital first customer experiences If youre ready for a new challenge and want to make a far-reaching impact through your work, this could be the opportunity youre looking for We're offering this role at vice president level What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and se...

Required Skills Strong expertise in Azure Data Factory (ADF) pipelines, dataflows, triggers, integration runtime. Solid understanding of ETL concepts , data warehousing, and data modeling. Hands-on experience with SQL/T-SQL , performance tuning, stored procedures. Experience working with Azure Data Lake Storage (ADLS) and Azure DevOps (CI/CD for data pipelines). Knowledge of Azure Synapse , Databricks , or PySpark is a plus. Good understanding of JSON, REST APIs, and integration patterns. Strong debugging, problem-solving, and analytical skills. Good to Have Experience with ADF Mapping Data Flows and Wrangling Data Flows . Experience with Power BI , Data Bricks , or Azure Functions . Knowled...

As an AI Engineer with Python at Artmac Soft, your role involves utilizing your 0-1 year of experience in AI engineering with a focus on Python programming. You are expected to have knowledge of Cursor AI, Render, and CloudFlare, along with proficiency in Python libraries and frameworks like TensorFlow, PyTorch, sci-kit-learn, NumPy, and pandas. Your responsibilities include a strong understanding of machine learning algorithms and deep learning architectures, experience with cloud platforms (AWS, GCP, Azure), and containerization tools such as Docker and Kubernetes. You will be working on CI/CD pipelines, MLOps practices, NLP, computer vision, distributed computing, big data technologies, a...

We are looking for a Sr. Kafka Engineer with 8+ years of experience. Responsibilities Design, develop, and manage Kafka-based data pipelines Monitor and optimize Kafka clusters Troubleshoot Kafka related issues Handle customer queries and support Requirements Minimum 8 years of technical development experience 3+ years of experience in Kafka

Define organizational AI strategy Lead a dynamic team and work in a fast paced environment Job Description As the AI lead, you will be responsible for - . Define and execute the AI / GenAI strategy for Operations, Risk, Underwriting & Claims, and Customer Service functions. . Partner with business heads to identify high-value GenAI use cases and prioritize by ROI, speed-to-value and regulatory sensitivity. . Act as the chief evangelist for Generative AI - educate stakeholders, run pilots, and drive scaled adoption while ensuring safety and compliance. . Establish robust frameworks to track and communicate the value and ROI of AI initiatives to all stakeholders. . Lead design and delivery of ...

ZS is a place where passion changes lives. As a management consulting and technology firm focused on improving life and how we live it, we transform ideas into impact by bringing together data, science, technology and human ingenuity to deliver better outcomes for all. Here you'll work side-by-side with a powerful collective of thinkers and experts shaping life-changing solutions for patients, caregivers and consumers, worldwide. ZSers drive impact by bringing a client-first mentality to each and every engagement. We partner collaboratively with our clients to develop custom solutions and technology products that create value and deliver company results across critical areas of their busines...

Job Description We are seeking an experienced AI/ML Engineer to lead the design and implementation of AI-powered solutions leveraging large language models (LLMs), multimodal AI, and advanced generative technologies. You will play a pivotal role in defining the technical vision, architecture, and strategy for enterprise-grade GenAI systems integrating them seamlessly into existing data, application, and cloud ecosystems. The ideal candidate combines deep technical expertise in AI/ML with strong solution design, stakeholder management, and hands-on implementation experience. Key Responsibilities Design and architect end-to-end Generative AI solutions (text, image, speech, code, etc.) that mee...

Key Responsbilities Design and implement data pipelines for supply chain data (e.g., inventory, shipping, procurement). Develop and maintain data warehouses and data lakes. Ensure data quality, integrity, and security. Collaborate with supply chain stakeholders to identify analytics requirements. Develop data models and algorithms for predictive analytics (e.g., demand forecasting, supply chain optimization). Implement data visualization tools (e.g., Tableau, Power BI). Integrate data from various sources (e.g., ERP, PLMs, other data sources). Develop APIs for data exchange. Work with cross-functional teams (e.g., supply chain, logistics, IT). Communicate technical concepts to non-technical ...

Role Overview: You will be responsible for designing and building data pipelines, Lakehouse architectures (BronzeSilverGold), and semantic models in Microsoft Fabric. Your role will involve developing data ingestion and transformation workflows using Data Factory, Dataflows Gen2, SQL, and PySpark. Additionally, you will integrate Fabric with Power BI for analytics and reporting, ensuring data quality, governance, and performance optimization. It will be crucial for you to implement DevOps (CI/CD, Git) practices for Fabric assets and stay updated with Microsoft Fabric roadmap (Cortex AI, Copilot, Real-Time Intelligence). Key Responsibilities: - Design and build data pipelines, Lakehouse archi...

Overview In this role, you will play a key role in automating and streamlining the data processing pipeline for AMESA markets. Responsibilities Own data pipeline development end-to-end, spanning data modeling, testing, scalability, operability and ongoing metrics. Ensure that we build high quality software by reviewing peer code check-ins Define best practices for product development, engineering, and coding as part of a world class engineering team in Hyderabad Focus on delivering high quality data pipelines and tools through careful analysis of system capabilities and feature requests, peer reviews, test automation, and collaboration with QA engineers Develop software in short iterations t...

What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and secure. Youll drive customer value by understanding complex business problems and requirements to correctly apply the most appropriate and reusable tool to gather and build data solutions. Youll support our strategic direction by engaging with the data engineering community to deliver opportunities, along with carrying out complex data engineering tasks to build a scalable data architecture. Your responsibilities will al...

Skills: Advanced proficiency in Azure Data Factory and related Azure services for designing ETL workflows. Strong understanding of data warehouse design, specifically in the context of Azure Synapse Analytics. Proficient in Python for automating data processing tasks and integrating with APIs as needed. Familiarity with PowerBI for building and maintaining reports and dashboards. Understanding of cloud architecture and capabilities, particularly in a Microsoft Azure environment. Responsibilities: Design, implement, and maintain data pipelines, ensuring data accuracy and consistency. Collaborate with stakeholders in both reporting and analytics to gather requirements and provide insights. Mon...

As a Technical Lead - Cloud Data Platform (Azure) at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team to e...