3177 Data Pipelines Jobs - Page 9

Job Purpose and Impact The Senior Consultant ?? Data & Analytics Reporting is a lead contributor responsible for leading the design, development, and delivery of advanced analytics solutions. This role involves managing complex datasets, building scalable reporting frameworks, and delivering actionable insights that drive business decisions. The individual playing this role takes directions from the Supervisor/Manager of this team, partners with cross-functional teams to ensure data integrity, applies advanced statistical techniques, and mentors junior analysts in best practices for data visualization and reporting. The role also includes leadership responsibilities such as guiding analytics...

We are looking for a skilled Senior Data Engineer with 5 to 10 years of experience to join our team in Gurugram. The ideal candidate will have expertise in building Data Lakes, Data Warehouses using Big data & cloud stack, and data modeling. Roles and Responsibility Design and develop scalable data pipelines and architectures. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems and databases. Ensure data quality and integrity through data validation and testing procedures. Implement Agile development methodologies to ensure timely delivery of projects. Optimize data storage and retrieval processes for improved...

Highly skilled Data Delivery Developer with expertise in SQL, Snowflake, and AWS infrastructure. The ideal candidate will have hands-on experience in complex data migration projects, particularly migrating from SSIS/SQL Server to Snowflake, and a strong understanding of cloud resource management on AWS. Define challenges of large and complex migration projects to best practices and have aptitude of high-level design thinking to build scalable solutions. Key Responsibilities Design, develop, and maintain scalable data pipelines using Snowflake and AWS services. Write complex and optimized SQL queries for data extraction, transformation, and reporting. Automate data workflows and delivery proc...

We are seeking a highly skilled Lead Developer to serve as the primary technical point of contact for our Royalty/Rights Management and Revenue systems. This role is critical for bridging technology and business, ensuring seamless coordination with stakeholders while delivering robust, scalable solutions. The ideal candidate will have deep expertise in data integration, cloud technologies, analytics, SQL development, and application maintenance , combined with strong communication and leadership skills. In addition to managing integrations and analytics, this individual will also oversee several legacy databases and data marts , and develop applications using MySQL and PostgreSQL . Key Respo...

Asset Management and Configurator Required Qualifications Extensive experience with Flexera (platform administration, agent/beacon management, support ticket workflows). Proven track record in L2/L3 support roles, preferably within large-scale enterprise environments. Strong troubleshooting and problem-solving skills. Excellent communication and coordination abilities, especially when working with cross-functional technical teams. Experience managing support cases and working within defined SLAs. Ability to navigate and administer the Flexera portal with confidence. Familiarity with internal tools/services and data pipelines is a plus. Mandatory Skills : ServiceNow Core . Experience : 5-8 Ye...

Job Purpose and Impact The Professional, Data Engineering job designs, builds and maintains moderately complex data systems that enable data analysis and reporting. With limited supervision, this job collaborates to ensure that large sets of data are efficiently processed and made accessible for decision making. Key Accountabilities DATA & ANALYTICAL SOLUTIONS: Develops moderately complex data products and solutions using advanced data engineering and cloud based technologies, ensuring they are designed and built to be scalable, sustainable and robust. DATA PIPELINES: Maintains and supports the development of streaming and batch data pipelines that facilitate the seamless ingestion of data f...

Position Description: Shift : General Education Qualification : Bachelors degree in computer science or related field or higher with minimum 2 years of relevant experience We are seeking a proactive and technically skilled engineer with hands-on experience in at Milvus vector database Engineer The ideal candidate should have 2 to 7years of industry experience and possess the ability to collaborate in a dynamic and fast-paced environment. Your future duties and responsibilities: Design, implement, and maintain Milvus vector database solutions for large-scale similarity search Collaborate with ML teams to store and query vector embeddings from NLP, CV, and multimodal models Optimize database p...

Job description We are seeking a highly skilled and motivated Big Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience with Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java. You will be responsible for designing, developing, and optimizing scalable data pipelines and big data solutions to support analytics and business intelligence initiatives. Qualification We are seeking a highly skilled and motivated Big Data Engineer to join our data engineering team The ideal candidate will have hands-on experience with Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java...

Job description Design and execute test plans for ETL processes, ensuring data accuracy, completeness, and integrity. Develop automated test scripts using Python or R for data validation and reconciliation. Perform source-to-target data verification, transformation logic testing, and regression testing. Collaborate with data engineers and analysts to understand business requirements and data flows. Identify data anomalies and work with development teams to resolve issues. Maintain test documentation, including test cases, test results, and defect logs. Participate in performance testing and optimization of data pipelines. Required Skills & Qualifications: Strong experience in ETL testing acr...

Role Overview: As a candidate for this role, you would be responsible for the following: Key Responsibilities: - Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. - Experience with Spark optimization techniques. - Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning. - Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer. - Proficiency in Python/Scala/Java for Spark development and integrate with ETL process. - Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database). - Understanding of data quality best practices and dat...

As an experienced candidate with over 8 years of overall experience and 5+ years of relevant experience, you will be responsible for the following key responsibilities: - Strong understanding of Spark core including RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming - Proficiency in Delta Lake features such as time travel, schema evolution, and data partitioning - Designing and building data pipelines using Spark and Delta Lake - Solid experience in Python/Scala/Java for Spark development - Knowledge of data ingestion from files, APIs, and databases - Familiarity with data validation and quality best practices - Working knowledge of data warehouse concepts and data modeling - Hands-on...

Join us as an Analyst Data operations at Barclays, where you'll spearhead the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionize our digital offerings, ensuring unapparelled customer experiences. To be successful as an Analyst Data operations, you should have experience with: B.Com (Pass/Hons)/MBA or equivalent with minimum 6 Years of working experience in Finance/ Banking/ Reconciliation or similar operations. Reasonable Knowledge of Financial Statements and Services in banking environment. Basic understanding of Reference data management and SAP elements to perform impact assessment required from new projects and c...

Join us as Assistant Vice President Data Analyst to enable effective data governance, risk management, and compliance within the bank, acting as a liaison between business units and the Data & Records Management (DRM) function, translating business needs into actionable strategies and ensuring efficient implementation of DRM. To be successful in this role, you should Banking industry experience, ideally within large matrixed organisations. Good knowledge of data and records governance frameworks, data risk management and regulatory expectations. Previous experience in operating data and/or records governance controls. Ideally, previous experience in supporting the operation of risk and contr...

Reference 25000MV0 Responsibilities We are seeking an experienced Senior Production Support Engineer with strong expertise in Sprak, SCALA, SNOWFLAKE, Unix, DevOps tools, and distributed computing environments. The ideal candidate will have hands-on experience supporting Big Data ecosystems and cloud-based platforms, ensuring the stability, performance, and reliability of critical business applications. Responsibilities Provide end-to-end production support for large-scale data processing and analytics applications running on distributed systems. Monitor, troubleshoot, and resolve issues in real-time across Unix-based and Big Data environments. Support and maintain data pipelines and workflo...

About Track3D Track3D is an AI-driven Reality Intelligence platform built for the construction industry. By unifying reality capture data from drones, 360 cameras, and mobile scans, Track3D enables automated progress tracking, early deviation detection, and real-time project visibility helping teams reduce rework and deliver projects with confidence We are a fast-growing enterprise SaaS startup working with some of the biggest names in the construction industry and we're just getting started. Role Overview We're looking for a Sales Operations Manager (RevOps) who brings structure and clarity to revenue data across Sales, Marketing, and Customer Success at Track3D. This role sits at the cente...

Description Translation Services seeks a passionate Machine Learning Engineer (MLE) to drive innovations in image translation technology to reach every Amazon customer in their own language.Are you excited about pushing the boundaries of computer vision and language translation at massive scale Do you thrive in environments where you can lead innovation in Visual Language Models If so, we're seeking a Machine Learning Engineer to join the Translation Services (TS) India Science team.TS provides technology solutions and services that deliver quality translations at the lowest possible cost for Amazon businesses. As our MLE, you'll be at the forefront of developing and deploying state-of-the-a...

Role Overview: As a Senior Data Engineer at our company, your primary responsibility will be to support enterprise data modernization and analytics transformation initiatives, focusing on Microsoft Fabric. Your expertise in data pipelines, lakehouse architecture, and experience in modernizing legacy data systems to cloud-native solutions will be crucial in building scalable, secure, and high-performing data solutions. You will work closely with Data Architects, BI Developers, and Analysts to deliver scalable data models for analytics and reporting. Key Responsibilities: - Design and implement data pipelines using Microsoft Fabric's Data Factory, Synapse Data Engineering, and OneLake componen...

As a Data Pipeline Tester, your main responsibility is to ensure the integrity and performance of data pipelines within the organization. You will collaborate closely with data engineers, data scientists, and other stakeholders to develop and implement robust testing strategies. Your efforts will be focused on identifying and resolving issues to uphold data quality and enhance pipeline efficiency. Key Responsibilities: - Proven experience in roles such as data pipeline tester, data quality analyst, or similar positions. - Demonstrated strong understanding of data pipelines, ETL processes, and data transformation. - Proficiency in utilizing data querying and scripting languages like SQL, Pyth...

As a Technical Trainer, you will be responsible for designing, developing, and delivering engaging training sessions in the domains of Python Programming, Java Development, Data Science, Microsoft Azure, and Data Engineering. Your role involves creating effective training content, conducting online/offline sessions, and providing real-time examples to enhance understanding. Additionally, you will guide learners through assessments, case studies, and projects, monitor their progress, and offer feedback to ensure clarity of concepts. It is crucial to stay updated with current technologies and industry trends, as well as address student doubts effectively. Key Responsibilities: - Design, develo...

Role Overview: As a Data Engineer, your role involves designing, building, and maintaining the company's data infrastructure, which includes databases and data pipelines. You will be responsible for ensuring that data is easily accessible and usable for data scientists, analysts, and other stakeholders. Your main tasks will include developing and implementing systems for data collection, storage, processing, and analysis. Key Responsibilities: - Designing and building data pipelines: Creating efficient and scalable systems for extracting, transforming, and loading (ETL) data from various sources into data warehouses or other storage systems. - Developing and maintaining databases and data wa...

As a Senior Data Engineer at ACV, a technology company revolutionizing the automotive industry, you will be a core member of the engineering team in Chennai, India. Your role will be crucial in improving the technical quality of data-focused products, driving innovation, and supporting business growth. - Collaborate with data team members across different locations to deliver new data pipelines and actionable insights - Work closely with the engineering and business teams to shape the technical design of platforms and solve complex problems - Contribute to evolving the tech stack over time and explore new technologies to solve upcoming challenges Qualifications Required: - Experience working...

Role Overview: As a Data Architect with Generative AI expertise, your primary responsibility is to design and implement robust data architectures that support AI and machine learning (ML) workloads, specifically focusing on Generative AI applications. Your tasks will involve developing and optimizing data pipelines for efficiently and securely training, validating, and deploying AI models. Additionally, you will be integrating AI frameworks into data platforms to ensure scalability, low latency, and high throughput. Collaboration with data scientists, AI engineers, and stakeholders is essential to align data strategies with business goals. Leading initiatives to ensure data governance, secur...

As a Data Engineer with a background in property & casualty insurance, your role will involve supporting the creation of a second workstream within the Guidewire DataMart. This position is particularly well-suited for candidates with a consultant background. Key Responsibilities: - Utilize at least 5+ years of experience in data engineering, leveraging tools such as Guidewire, Azure, and Databricks. - Demonstrate familiarity with data lakes, Python programming, and deploying to Databricks. - Apply your full scope property & casualty insurance experience to handle data management and analysis specific to this sector. - Utilize your familiarity with Synapse to enhance data engineering processe...

Position Description: Azure data bricks developer with 3-6 years of experience We are seeking a skilled Azure Databricks Developer to design, develop, and optimize big data pipelines using Databricks on Azure. The ideal candidate will have strong expertise in PySpark, Azure Data Lake, and data engineering best practices in a cloud environment. Key Responsibilities: Design and implement ETL/ELT pipelines using Azure Databricks and PySpark Work with structured and unstructured data from diverse sources (e g , ADLS Gen2, SQL DBs, APIs) Optimize Spark jobs for performance and cost-efficiency Collaborate with data analysts, architects, and business stakeholders to understand data needs Develop re...

Position Description: Company Profile: Job Title: GenAI Solution Architect / Lead GenAI Engineer Position: Lead Analyst Experience: 6 Plus Category: Software Development Main location: Bangalore Position ID: J1025-1909 Employment Type: Full Time Your future duties and responsibilities: Job Summary: We are seeking a highly skilled and innovative GenAI Solution Architect / Lead AI Engineer to design, develop, and optimize Generative AI applications and system architectures. The ideal candidate will have extensive experience in Python, Azure Cloud services, and LLM frameworks (such as vLLM). This role requires a strong foundation in system design, performance optimization (throttling, queuing, ...