3177 Data Pipelines Jobs - Page 10

DBT responsibilities include designing, developing, and handling technical architecture, data pipelines, and performance scaling using tools to integrate Talend data and ensure data quality in a big data environment The candidate must be very strong in PL/SQL, including queries, procedures, and JOINs Experience in Snowflake SQL, writing SQL queries against Snowflake, and developing scripts using Unix, Python, etc , to perform Extract, Load, and Transform operations is essential It is good to have knowledge and hands-on experience with Fivetran Candidates who have worked in production support would be preferred Hands-on experience with Snowflake utilities such as SnowSQL, SnowPipe, Python, Ta...

About this role: Wells Fargo is seeking a Lead Quantitative Model Solutions Specialist . In this role, you will: Lead complex, large-scale model maintenance, optimization, and planning initiatives related to operational processes, controls, reporting, testing, implementation, and documentation Review and analyze complex multi-faceted model operations and optimization challenges that require in-depth evaluation of multiple factors including intangibles or unprecedented factors Develop model processes and optimization strategies for short- and long-term objectives; support and provide insights regarding a wide array of business initiatives Make decisions in complex and multi-faceted situations...

Roles and Responsibilities: Resource is responsible for designing, developing, and maintaining data integration solutions using the Oracle Data Integrator platform. This role involves transforming raw data into actionable insights to support business intelligence and data warehousing objectives. Design, develop, implement, and optimize data integration processes using Oracle Data Integrator (ODI). The resource will collaborate with cross-functional teams to translate business requirements into technical specifications and ensure data accuracy, integrity, and performance across all data pipelines. 5 years of proven experience in Oracle Analytics Cloud (OAC) development, RPD development & ODI ...

As Lead PM , you will drive product security (audit logging, robust roles/permissions design including Partner models), as well as key collaboration features like notifications and notes. You will execute the product roadmap hands-on, translating requirements into impactful, compliant, and user-friendly capabilities Key Responsibilities: Lead roadmap, design, and delivery for : Audit Logs: End-to-end ownership, including event capture, tamper-proof storage, search/filter, and rich reporting/analytics. Roles & Permissions: Design and evolve granular role-based access control (RBAC) for users and complex partner scenarios Notifications & Notes: Define and deliver contextual notifications (in-a...

Mandatory skill AWS Glue, Redshift, Python/Pyspark, SQL, ETL.Design and build scalable data pipelines using AWS services like AWS Glue, Amazon Redshift, and S3.Develop efficient ETL processes for data extraction, transformation, and loading into data warehouses and lakes. Create and manage applications using Python, SQL, Databricks, and various AWS technologies.Automate repetitive tasks and build reusable frameworks to improve efficiency. Skill Proficiency Level expected AWS Data Engineer - AWS Glue, Amazon Redshift, S3Expert with 8.5 to 10+ years of hands-on experienceETL Process, SQL, Databricks

Bachelors degree or equivalent in Computer Engineering Computer Science or a related field 5 years of experience in data software engineering role 3 years of strong hands on experience with Databricks with AWS as a cloud platform which including Spark based processing data pipelines and the platforms tools for analytics and machine learning 2 years of experience with building AWS cloud based data pipelines and AI solutions 2 years of Hands on experience with Python and PySpark The role holder will possess a blend of design skills needed for Agile data development projects Proficiency or demonstratable passion for learning in data engineer techniques and testing methodologiesStrong written an...

We are looking for a skilled Data Engineering Manager with 8 to 13 years of experience to lead our team in designing, developing, and optimizing scalable data pipelines and infrastructure. This position is based in Hyderabad. Roles and Responsibility Lead a team of data engineers to design, develop, and optimize scalable data pipelines and infrastructure. Design, develop, and maintain scalable ETL (Extract, Transform, Load) pipelines for efficient data processing and integration. Build and maintain scalable, fault-tolerant data architectures ensuring efficient data storage, retrieval, and processing across cloud and on-premises environments. Oversee the design, implementation, and performanc...

About Paytm: Paytm is India''s leading mobile payments and financial services distribution company. Pioneer of the mobile QR payments revolution in India, Paytm builds technologies that help small businesses with payments and commerce. Paytm - - s mission is to serve half a billion Indians and bring them to the mainstream economy with the help of technology. About the Role: As a Director of Product for Central Data Platforms you play a pivotal role in establishing a robust data engine catering to both internal as well as external stakeholders covering multiple technologies, products and systems. You will collaborate with leadership and cross-functional partners in driving the conceptual and ...

About The Role Technical Lead Location Chennai Experience 12+ years (with at least 3 years in technical leadership roles) Role Summary As a Technical Lead , you will own end-to-end technical delivery for fleet- and vessel-level solutions in the maritime domain. This role combines architectural leadership with hands-on engineering across React, Python, AWS, Snowflake, and Kafka , ensuring secure, scalable, and high-performing solutions. You will lead a team, collaborate with domain experts, and drive innovation in real-time telemetry, data platforms, and model operationalization. Key Responsibilities Define and evolve architecture for frontend, backend, data pipelines, and edge gateway patter...

Snowflake Developer ADF Developer ADF & Snowflake (2- 4 Years Experience) Key Technologies Azure Data Factory (ADF), Snowflake, SQL, Azure Services Job Summary We are seeking a motivated and detail-oriented Data Engineer with 2- 4 years of experience in Azure Data Factory and Snowflake SQL to join our growing data engineering team. You will be responsible for building, maintaining, and optimizing data pipelines and integrating data from various sources to Snowflake. The ideal candidate has hands-on experience in developing ETL/ELT processes using ADF and writing efficient Snowflake SQL queries. Key Responsibilities : Design, develop, and maintain scalable data pipelines using Azure Data Fact...

We are seeking an experienced Data Engineer / Analytics Engineer with a strong background in SQL, ETL development, and cloud-based data platforms The ideal candidate will have a solid foundation in data warehousing, data transformation, and analytics engineering practices This role involves building scalable data pipelines, ensuring data quality, transforming complex datasets, and collaborating closely with both technical teams and business stakeholders Attention to detail, strong communication skills, and a structured coding approach are essential for success

Bachelors degree or equivalent in Computer Engineering Computer Science or a related field 5 years of experience in data software engineering role 3 years of strong hands on experience with Databricks with AWS as a cloud platform which including Spark based processing data pipelines and the platforms tools for analytics and machine learning 2 years of experience with building AWS cloud based data pipelines and AI solutions 2 years of Hands on experience with Python and PySpark The role holder will possess a blend of design skills needed for Agile data development projects Proficiency or demonstratable passion for learning in data engineer techniques and testing methodologies Strong written a...

Assist in building and testing AI models for knowledge generation and resolution recommendation. Develop scripts and connectors for ITSM tool integration. Support data preparation and model training activities. Document workflows and assist in creating knowledge artifacts. Participate in validation workshops with SMEs. Required Skills: Exposure to LLMs and AI driven automation. Basic understanding of ITIL processes and service desk operations. Knowledge of Python and API integration. Solid experience with LLMs, embeddings, RAG pipelines for IT Ops use cases. Strong Python coding skills with ability to build integrations, data pipelines, and automation scripts. Good understanding of ServiceNo...

Strong proficiency in Python, including experience with data manipulation libraries like Pandas, NumPy, and PySpark . Strong in writing and optimizing complex SQL queries for data retrieval, transformation, and analysis within Snowflake and PostgreSQL Design, implement, and optimize end-to-end data pipelines using Python, including tasks like data ingestion, transformation, and loading Integrate data from various sources, including databases, APIs, and streaming platforms. Preferable to have experience in Node.js and Angular Experience in working with VS Code or Spyder or similar IDEs Experience in working with Azure Storage accounts, azure identities Knowledge in working with Git and creati...

Design, develop, and maintain data pipelines and ETL processes using AWS and Snowflake. Implement data transformation workflows using DBT (Data Build Tool). Write efficient, reusable, and reliable code in Python. Optimize and tune data solutions for performance and scalability. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Ensure data quality and integrity through rigorous testing and validation. Stay updated with the latest industry trends and technologies in data engineering. Bachelor''s or Master''s degree in Computer Science, Engineering, or a related field. Proven experience as a Data Engineer or similar role. S...

Design, develop, and maintain data pipelines and ETL processes using AWS and Snowflake. Implement data transformation workflows using DBT (Data Build Tool). Write efficient, reusable, and reliable code in Python. Optimize and tune data solutions for performance and scalability. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Ensure data quality and integrity through rigorous testing and validation. Stay updated with the latest industry trends and technologies in data engineering. Bachelor''s or Master''s degree in Computer Science, Engineering, or a related field. Proven experience as a Data Engineer or similar role. S...

Job Summary: We are looking for a skilled AWS Data Engineer with strong experience in building and managing cloud-based ETL pipelines using AWS Glue, Python/ PySpark , and Athena, along with data warehousing expertise in Amazon Redshift. The ideal candidate will be responsible for designing, developing, and maintaining scalable data solutions in a cloud-native environment. Design and implement ETL workflows using AWS Glue, Python, and PySpark . Develop and optimize queries using Amazon Athena and Redshift. Build scalable data pipelines to ingest, transform, and load data from various sources. Ensure data quality, integrity, and security across AWS services. Collaborate with data analysts, da...

Senior ETL QA (Data, Power BI) Role Overview The Onsite Senior ETL QA will lead quality assurance efforts for data platforms and Power BI solutions, ensuring accuracy, reliability, and compliance across MDM, ETL, and reporting layers. This role involves defining test strategies, mentoring junior QA engineers, and collaborating with onshore teams to deliver high-quality solutions in a global delivery model. Key Responsibilities: Define and implement comprehensive QA strategies for data pipelines, MDM outputs, and Power BI dashboards. Lead functional, regression, and performance testing for large-scale data and BI solutions. Validate complex data transformations, DAX calculations, KPIs, and vi...

We are looking for a Senior Big Data Engineer with deep expertise in large-scale data processing technologies and frameworks. This is a remote, contract-based position suited for a data engineering expert with strong experience in the Big Data ecosystem including Snowflake (Snowpark), Spark, MapReduce, Hadoop, and more. Key Responsibilities Design, develop, and maintain scalable data pipelines and big data solutions Implement data transformations using Spark, Snowflake (Snowpark), Pig, and Sqoop Process large data volumes from diverse sources using Hadoop ecosystem tools Build end-to-end data workflows for batch and streaming pipelines Optimize data storage and retrieval processes in HBase, ...

About The Role Project Role : Program/Project Management Lead Project Role Description : Manage overall delivery of a program or project to achieve business outcomes. Define project scope and monitor execution of deliverables. Communicate across multiple stakeholders to manage expectations, issues and outcomes. Must have skills : Program Project Management Good to have skills : Microsoft SQL Server, Product Development Management, Microsoft Azure Data Services Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Program/Project Management Lead, you will manage the overall delivery of a program or project to achieve business outco...

About The Role Project Role : Technology Architect Project Role Description : Design and deliver technology architecture for a platform, product, or engagement. Define solutions to meet performance, capability, and scalability needs. Must have skills : Salesforce Technical Architecture Good to have skills : Salesforce Data, Cloud Data Architecture Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Salesforce Data Cloud Architect, you will be responsible for designing and implementing scalable, secure, and high-performance data architectures using Salesforce Data Cloud. You will work closely with cross-functional teams to define...

Job Title: Oracle Fusion Technical Lead Key Responsibilities Lead the end-to-end technical architecture, design, and delivery of Oracle Fusion Cloud solutions across Finance, Procurement, and FAH. Oversee development of RICEW components including integrations, reports, extensions, conversions, and workflows. Architect and guide the build of integrations using Oracle Integration Cloud (OIC) - REST/SOAP APIs, scheduled jobs, app-driven flows, and file-based integrations. Own the architecture, design, and governance of data pipelines leveraging FDI (Fusion Data Intelligence), ODI (Oracle Data Integrator), and ADW (Autonomous Data Warehouse) for analytics, reporting, and downstream data consumpt...

AI Architect Overview: The Generative AI Architect will lead the design, development, and deployment of enterprise-grade generative AI solutions leveraging LLMs, diffusion models, and multimodal architectures. This role combines deep technical expertise with solution architecture, governance, and strategic advisory skills to enable responsible and scalable adoption of GenAI across business domains. You will work cross-functionally with data scientists, software engineers, risk and compliance teams, and business stakeholders to define reference architectures, integrate with existing systems, and ensure secure, ethical, and compliant AI adoption. Responsibilities: Architecture & Solution Desig...

Your key responsibilities Technical Excellence Proficient in building and managing data pipelines, models, and integrations in accordance with Medallion Lake architecture using Databricks notebooks and scripting. Hands-on experience in supporting cloud-based analytics delivery, particularly with Azure and Databricks, while working with ERP systems such as SAP or Oracle. Strong understanding of enforcing data governance practices to ensure data integrity and compliance across various industries. Skilled in deriving actionable insights and enabling data-driven initiatives through strategic consulting and implementation of robust data processes. Experience in collaborating with cross-functional...

The opportunity Were looking for candidates with strong technology and data understanding in big data engineering space, having proven delivery capability. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data and Analytics team. Your key responsibilities Develop & deploy big data pipelines in a cloud environment using Azure Cloud services ETL design, development and migration of existing on-prem ETL routines to Cloud Service Interact with senior leaders, understand their business goals, contribute to the delivery of the workstreams Design and optimize model codes for faster execution Skills and attributes for success Overall 4+ years of IT experien...