3177 Data Pipelines Jobs - Page 7

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : AWS Redshift Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project mil...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Snowflake Data Warehouse Good to have skills : Oracle Procedural Language Extensions to SQL (PLSQL), Amazon Web Services (AWS), Data Warehouse ETL Testing Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Java Good to have skills : Java Enterprise Edition Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality...

Responsibilities: Assemble large, complex data that meet functional/non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing transformation for greater scalability, etc. Use the infrastructure/services required for optimal extraction, transformation, and loading of data from a wide variety of data sources using GCP and AWS services. Create data for analytics and CRM team members that assist them in building and optimizing our product. Work with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data requirement n...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Data Science Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and impleme...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google BigQuery Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and impl...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Data Science Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and impleme...

Company Description Cloudalytix is a cutting-edge consulting and delivery firm, specializing in cloud, data, and AI solutions for start-ups, SMBs, and large enterprises. Founded by industry veterans, the company bridges the gap between strategic advice and practical execution, enabling clients to achieve measurable results through digital transformation. Cloudalytix creates future-ready, secure, and scalable solutions with a strong emphasis on trust, innovation, and partnership. Cloudalytix serves diverse industries, including financial services, healthcare, manufacturing, retail, and emerging tech, striving to be the most trusted transformation partner in the industry. Role: Microsoft Fabri...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Alteryx Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Microsoft Azure Databricks Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions for data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : MicroStrategy Business Intelligence Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring d...

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Data Engineering Good to have skills : NA Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating with various teams to ensure ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : Python (Programming Language), Google BigQuery, PySpark Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creat...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Microsoft SQL Server Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Key ResponsibilitiesData Pipeline Development:Design and implement robust ETL/ELT pipelines using tools like Azure Synapse Pipeline & Notebooks to extract, transform, and load data from various sources.Data Storage Mana...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : Python (Programming Language), Google BigQuery, PySpark Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creat...

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title And Summary Senior Product Specialist - Technical Integrations Overview As a Senior Product Specialist -Technical Integration, you will be responsible for designing and implementing inte...

The Data Engineer engineers secure, scalable and reliable technology solutions, with minimal mentoring, to advance CMEG in the global marketplace and serve risk management needs of customers around the world. Key Responsibilities: Provide operational support for our analytics system built on the KDB+ database management system and tooling. This includes monitoring performance, troubleshooting issues, and maintaining data pipelines to ensure system stability. Actively participate in the design, development, and implementation of new analytics solutions on GCP. This will involve refactoring existing workflows and building new capabilities using native GCP services. Utilize GCP's real-time anal...

You bring extensive experience in DBT, including designing and developing technical architecture, data pipelines, and performance scaling You have leveraged tools to integrate Talend data and ensure data quality in a big data environment Your PL/SQL skills are very strong, particularly with queries, procedures, and JOIN operations You have hands-on experience with Snowflake, including writing SQL queries, developing scripts in Unix and Python to perform extract, load, and transform (ELT) operations Familiarity with Fivetran and hands-on experience with it is a plus Candidates who have worked in production support environments are preferred Your background includes direct experience with Snow...

Agentic AI Software Engineer to join our team and help build the next generation of intelligent agentdriven applications. This is a handson engineering role with a strong learning focus where you contribute to data pipelines RAG systems and AIenabled APIs while working alongside experienced engineers and technical leads. Assist in developing GenAI applications using modern frameworks LangChain Lang Graph AutoGen or AWS Bedrock Agents. Support the design and implementation of data ingestion pipelines for structured and unstructured data Contribute to retrievalaugmented generation RAG projects by building embeddings working with vector databases and integrating context into models. Integrate a...

P2-C3-STSJDlooking for Python engineers who have experience in the following:A Python Developer designs, builds, and maintains software using the Python language, focusing on writing efficient, reusable server-side code, creating APIs, and managing data pipelines Key responsibilities include full-stack development, coding for web apps and data-intensive tools, collaborating with cross-functional teams, integrating third-party services, and ensuring code quality and scalability Skills like proficiency in Python frameworks (e g , Django, Flask), understanding databases, and knowledge of DevOps practices are essential Secondary skills JavaExperience in Commercial Banking You should care about P...

Candidate needs to have exposure as a Developer, Support Engineer & Testing Candidate must have knowledge on either platforms like Informatica, Ab Initio, DataStage OR custom code using PythonDesign and execute test plans for ETL processes, ensuring data accuracy, completeness, and integrity Develop automated test scripts using Python or R for data validation and reconciliation Perform source-to-target data verification, transformation logic testing, and regression testing Collaborate with data engineers and analysts to understand business requirements and data flows Identify data anomalies and work with development teams to resolve issues Maintain test documentation, including test cases, t...

We are seeking a Reltio Senior System Architect with deep expertise in AWS cloud services and Databricks based Lakehouse architectures to lead enterprise scale Master Data Management MDM solutions This role will own the end to end architecture integrating Reltio MDM Databricks and AWS native services enabling trusted master data across analytics AI and operational systems Required Skills Qualifications 10 years of experience in enterprise data architecture and integration5 years of hands on Reltio MDM experience including match merge survivorship and complex entity modeling Strong expertise in Databricks Lakehouse architecture includingPySpark and Spark SQLDelta Lake ACID time travel optimiz...

Bachelors degree or equivalent in Computer Engineering Computer Science or a related field 5 years of experience in data software engineering role3 years of strong hands on experience with Databricks with AWS as a cloud platform which including Spark based processing data pipelines and the platforms tools for analytics and machine learning 2 years of experience with building AWS cloud based data pipelines and AI solutions 2 years of Hands on experience with Python and PySpark The role holder will possess a blend of design skills needed for Agile data development projects Proficiency or demonstratable passion for learning in data engineer techniques and testing methodologiesStrong written and...

AWS Data Engineer Mandatory skill- AWS Glue, Redshift, Python/Pyspark, SQL, ETL Design and build scalable data pipelines using AWS services like AWS Glue, Amazon Redshift, and S3 Develop efficient ETL processes for data extraction, transformation, and loading into data warehouses and lakes Create and manage applications using Python, SQL, Databricks, and various AWS technologies Automate repetitive tasks and build reusable frameworks to improve efficiency Skill Proficiency Level expectedAWS Data Engineer - AWS Glue, Amazon Redshift, S3Expert with 8 5 to 10+ years of hands-on experienceETL Process, SQL, Databricks

About the Role We are looking for experienced data engineers who can independently build, optimize, and manage scalable data pipelines and platforms. In this role, youll work closely with clients and internal teams to deliver robust data solutions that power analytics, AI/ML, and operational systems. Youll also help mentor junior engineers and bring engineering discipline into our data engagements. Key Responsibilities Design, build, and optimize large-scale, distributed data pipelines for both batch and streaming use cases. Implement scalable data models, data warehouses/lakehouses, and data lakes to support analytics and decision-making. Collaborate with cross-functional stakeholders to un...