1617 Data Pipelines Jobs - Page 5

As a Senior Cloud Data Platform (Azure) Engineer at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team to en...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

The Data Engineer exercises judgment when following general instructions and works with minimal instruction to support the integration and automation of data solutions. This role focuses on data massaging, reconciliation, and analysis, resolving routine to semi-routine issues. Responsibilities include creating optimized SQL queries, managing data pipelines, and collaborating with cross-functional teams to ensure data accuracy and availability. What will you do: Write optimized and scalable complex SQL queries Automate data processing tasks using Python, focusing on cleaning and merging datasets. Manage data pipelines, including scheduling, monitoring, and debugging workflows. Collaborate wit...

Work with broader team to build, analyze and improve the AI solutions. You will also work with our software developers in consuming different enterprise applications Required education Bachelor's Degree Required technical and professional expertise Resource should have 5-7 years of experience. Sound knowledge of Python and should know how to use the ML related services. Proficient in Python with focus on Data Analytics Packages. Strategy Analyse large, complex data sets and provide actionable insights to inform business decisions. Strategy Design and implementing data models that help in identifying patterns and trends. Collaboration Work with data engineers to optimize and maintain data pip...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data PlatformResponsibilities:Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFSExperienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platformExperience in developing streaming...

Work with broader team to build, analyze and improve the AI solutions. You will also work with our software developers in consuming different enterprise applications Required education Bachelor's Degree Required technical and professional expertise Resource should have 5-7 years of experience. Sound knowledge of Python and should know how to use the ML related services. Proficient in Python with focus on Data Analytics Packages. Strategy Analyse large, complex data sets and provide actionable insights to inform business decisions. Strategy Design and implementing data models that help in identifying patterns and trends. Collaboration Work with data engineers to optimize and maintain data pip...

We're seeking a results-driven and collaborative Software Development Manager to lead the design and development of IBM Consulting Advantage Platform. As a management leader, you'll collaborate with peers and stakeholders to ensure business continuity. You'll also be responsible for building and leading an impactful team of Developers & QA engineers, focusing on software developments, productivity improvements and fostering a culture of continuous learning and improvement. In this role, you will be responsible for: * Lead a team of engineers to meet release dates along with committed deliverables on-time and with quality * Balance priorities and work assignments across team members following...

About the Job The Red Hat Experience Engineering (XE) team is looking for a skilled AI Technical Strategist/Software Engineer with excellent programming skills, deep familiarity with LLMs including practical experience with retrieval augmented generation (RAG) and agentic AI approaches. Must have hands-on experience in robust model evaluation, MLOps best practices and deployment, and the ability to partner with internal stakeholders. To be successful in this role, you should have an established set of foundational technical skills and the ability to learn quickly as we modernize platforms and tooling to demonstrate an AI-first approach. You will be accountable for helping to shape our techni...

Work with broader team to build, analyze and improve the AI solutions.You will also work with our software developers in consuming different enterprise applications Required education Bachelor's Degree Required technical and professional expertise Resource should have 5-7 years of experience. Sound knowledge of Python and should know how to use the ML related services.Proficient in Python with focus on Data Analytics Packages.Strategy Analyse large, complex data sets and provide actionable insights to inform business decisions.Strategy Design and implementing data models that help in identifying patterns and trends. Collaboration Work with data engineers to optimize and maintain data pipelin...

What youll be DOING What will your essential responsibilities include? Support pricing metrics reporting and ongoing integration projects. Manage quarterly/monthly/yearly financial data for MI reporting. Work with actuaries to refine rate monitor data anomalies by highlighting the outlier as well as accounts driving the deviations. Oversee the development and maintenance of complex data mappings that impact actuarial and finance data. Collaborate with IDA function to incorporate business requirements in reporting solutions. Partner with Finance, IT and Reserving functions to capture changes impact our data and processes to meet strategic goals. Demonstrate proactive communication with Busine...

Key Responsibilities: Data Testing Strategy & Execution: Design, develop, and execute comprehensive test plans and test cases for data-centric applications, ETL processes, data warehouses, data lakes, and reporting solutions SQL-Driven Validation: Utilize advanced SQL queries to perform complex data validation, data reconciliation, data integrity checks, and data quality assurance across various financial data sources ETL Testing: Conduct thorough testing of ETL (Extract, Transform, Load) processes, ensuring data is accurately extracted, transformed according to business rules, and loaded correctly into target systems Data Quality Assurance: Implement and monitor data quality checks, identif...

Role Description: We are seeking a seasoned Engineering Manager (Data Engineering) to lead the end-to-end management of enterprise data assets and operational data workflows. This role is critical in ensuring the availability, quality, consistency, and timeliness of data across platforms and functions, supporting analytics, reporting, compliance, and digital transformation initiatives. You will be responsible for the day-to-day data operations, manage a team of data professionals, and drive process excellence in data intake, transformation, validation, and delivery. You will work closely with cross-functional teams including data engineering, analytics, IT, governance, and business stakehold...

What you do We are seeking a seasoned Sr Manager of Data Engineering to drive the development and implementation of our data strategy for Manufacturing and Clinical Supply Chain. The ideal candidate possesses a strong blend of technical expertise and data-driven problem-solving skills. As a Sr. Manager Data Engineering, you will play a crucial role in leading, designing, building, and optimizing our data pipelines and platforms while mentoring junior engineers. Roles & Responsibilities: Contribute to the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions. Take ownership of data pipeline projects from inception to deployment, managing ...

What you will do In this vital role the Manager Field Reporting Build & Development, CBU / IBBU will be responsible for the full lifecycle of report development from identifying business needs through requirements elicitation, to designing and delivering robust, performant, fit-for purpose reporting solutions, to operations and support. This individual will oversee both front-end builds (dashboards, visualizations, reports, data user interfaces) and back-end builds (data models, pipelines, and integrations), ensuring scalable, reliable, and user-centric reporting. Furthermore, they will oversee operations and support of our reporting solutions ensuring data is available, up-to-date, and accu...

Job description: A minimum of 2-3 years experience in Azure Data Experience utilizing: Azure (Data Factory, Data Lake, Machine Learning, SQL Warehouse, Data Bricks), SAP Business Warehouse, Power BI, Business Objects (WEBI, Data Services, Excel Analysis) Working with streaming analytic tools like Azure Event Hubs, MuleSoft, and Logic Apps Building real-time data pipelines from data lake to warehouses and dashboards Working with tools like Azure Data Factory, Azure Data Bricks Architecting Data Hubs for MDM, Machine Learning, and Analytics Applications Working with Datahub platforms like Mark Logic or Azure Data Lake (Gen 1 or Gen 2) Providing feature pipelines for Machine Learning solutions ...

What you will do In this vital role you will maximizes domain and Software Development Lifecycle (SDLC) expertise to design, develop, and maintain software applications and solutions that meet business needs and ensure the availability and performance of critical systems and applications in the Learning & Development (L&D) functional area. This role is a key member of the L&D product team, and it involves working closely with product owners, developers, system administrators, and other engineers to create high-quality, scalable software solutions, automate operations, monitor system health, and respond to incidents to minimize downtime. Roles & Responsibilities: Work closely with product tea...

Job Summary Lead end-to-end digital initiatives that lift efficiency and quality across shopfloor and business functions. Own business analysis, project charters, and delivery while being hands-on with tech and strong on stakeholder/vendor management. Key Responsibilities Discover, scope, and justify projects; define KPIs and ROI, draft/own project charters. Lead solution design and development (.NET, Python, JavaScript, MS SQL); oversee web-portal development and internal dashboards. Drive IIoT/Industry 4.0 programs: sensor/PLC integration, OPC-UA/MQTT, edge gateways, data pipelines to MES/ERP/SCADA. Run Agile/Stage-Gate execution, budgets, risks, and change management. Manage vendors: RFPs...

Company Overview KKR is a leading global investment firm that offers alternative asset management as well as capital markets and insurance solutions. KKR aims to generate attractive investment returns by following a patient and disciplined investment approach, employing world-class people, and supporting growth in its portfolio companies and communities. KKR sponsors investment funds that invest in private equity, credit and real assets and has strategic partners that manage hedge funds. KKR's insurance subsidiaries offer retirement, life and reinsurance products under the management of Global Atlantic Financial Group. References to KKR's investments may include the activities of its sponsor...

Do you want to fine-tune OpenAI's most advanced models on Azure working with GPT-4.1, GPT-5, and beyond From supervised fine-tuning (SFT) to reinforcement learning methods like DPO and others, this role puts you hands-on with the techniques shaping the future of AI. On this team, you'll play a key role in improving the performance and reliability of customizing some of the world's most advanced AI models. You'll gain hands-on experience with fine-tuning workflows, experimenting with reinforcement learning techniques, and applying advanced optimization strategies that directly impact product quality. You'll also work directly with the training code, debug complex behaviors, and sharpen the sk...

Job Location HYDERABAD OFFICE APAC Job Description Procter & Gamble is one of the largest FMCG (Fast Moving Consumer Goods) company in the world with strong brands like Pampers, Ariel, Always, Gillette and Oral B just to name a few. For more information about P&G the company and our brands please visit www.pg.com and our career website at pgcareers.com Are you ready to unleash your technical creativity Innovation is the driving effort behind everything we do at P&G. Across the world, you will find thousands of scientists, engineers and developers working in manufacturing plants, innovation centers and distribution facilities. Overview Of The Job This role reports to IT Director and will be l...

Job Description Summary We are looking for a data-driven and strategic Technical Product Manager to lead the development and implementation of AI-powered pricing solutions in the healthcare domain. This role will drive the intersection of artificial intelligence and product strategy to deliver intelligent pricing models that improve affordability, access, and profitability across our healthcare offerings. You will work cross-functionally with engineering, pricing and finance teams to define and execute product roadmaps that leverage AI for dynamic and value-based pricing. You will work with the team and deliver a clear and descriptive set of requirements to enable the delivery of a products ...

Required Information Role** Microsoft Azure Data Engineer Required Technical Skill Set** SQL, ADF, ADB, ETL/Data background Desired Experience Range 4 Location of Requirement India Desired Competencies (Technical/Behavioral Competency) Must-Have** (Ideally should not be more than 3-5) Strong handson with Azure Data Factory (ADF), Azure Databricks, ADLS, SQL, ETL/ELT Pipelines building, orchestrating, and optimizing data pipelines. DevOps (version control (Git) Good-to-Have Water industry domain knowledge SN Responsibility of / Expectations from the Role 1 Deliver clean, reliable and scalable data pipelines 2 Ensure data availability and quality 3 Excellent communication and documentation abi...

The AI Platform organization at Microsoft builds the end-to-end Azure AI stack/PaaS and is core to Azure's innovation and differentiation, as well as all of Microsoft's flagship products, from Office to Teams, to Xbox. We are the team building Azure OpenAI, Azure ML, Cognitive Services, and the global Azure AI infrastructure for running the largest AI workloads on the planet. Within AI Platform, the AI Foundry team enables data scientists and developers to quickly and easily build, train, deploy, manage, and consume machine learning model. Our AI Foundry Finetuning Team is at the forefront of this mission, working on groundbreaking projects for finetuning of Azure OpenAI and OSS models. We c...

The ideal candidate will have 8+ years of experience in designing and implementing scalable data engineering systems. They should have a proven track record of delivering robust and efficient solutions in a fast-paced environment. Key Responsibilities Include Designing and implementing scalable and efficient architectures for ELT systems and data pipelines Leading the design of RESTful APIs to facilitate seamless integration and interaction with data systems Recommending tools, frameworks, and platforms (i.e., Snowflake, Redshift, PostgreSQL) to meet current and future business needs Working closely with data engineers, software developers, and stakeholders to translate business requirements...

ABOUT THE ROLE We are seeking a seasoned Senior Data Engineer to drive the development and implementation of our data strategy. The ideal candidate possesses a strong blend of technical expertise and data-driven problem-solving skills. As a Senior Data Engineer, you will play a crucial role in designing, building, and optimizing our data pipelines and platforms while mentoring junior engineers. Roles & Responsibilities: Contribute to the design, development, and implementation of automations to monitor our data and cloud platforms for reliability, cost, and maintenance. Take ownership of platform management, managing scope, timelines, and risks. Ensure data quality and integrity through rigo...