757 Data Pipelines Jobs - Page 16

Experience: 5-8 Years Work Mode: Pune & Hyderabad Job Type: Fulltime Mandatory Skills: Python, Pyspark, SQL, Snowflake, Airflow, ETL, Data Pipelines, Elastic Search, or AWS. Role Overview We are looking for a talented and passionate Senior Data Engineer to join our growing data team. In this role, you will play a key part in building and scaling our data infrastructure, enabling data-driven decision-making across the organization. You will be responsible for designing, developing, and maintaining efficient and reliable data pipelines for both ELT (Extract, Load, Transform) and ETL (Extract, Transform, Load) processes. Responsibilities Design, develop, and maintain robust and scalable data pi...

Role- Data Engineer Location: Bengaluru Hybrid Experience: 5-9 years Employment Type: Full-time Key Responsibilities Design, build, and optimize data pipelines using PySpark and Apache Airflow for large-scale ETL processes. Develop and maintain big data solutions in Python , ensuring scalability and efficiency. Write, optimize, and tune complex Oracle SQL / PL-SQL queries for data management and analytics. Implement and maintain data warehouse solutions (star/snowflake schema, facts/dimensions, SCDs). Translate banking domain use cases (e.g., NPA, provisioning, regulatory reporting) into scalable data pipelines. Monitor, troubleshoot, and optimize workflows, batch jobs, and data accuracy. Co...

DESCRIPTION Would you like to work on one of the world's largest transactional distributed systems How about working with customers and peers from the entire range of Amazon's business on cool new features Whether you're passionate about building highly scalable and reliable systems or a professional who likes to solve business problems, eCS (e-Commerce Services) is the place for you. eCS team constitutes of Customer identity services, Tax management and calculation services, Cart-Order services, Shipping/Region authority services and Accounting services. Within eCS our team (Data engineering and Analytics) is responsible for building data governance, security, authoring data pipelines, sett...

Job Description: This role, part of the Digital Intelligence team, offers opportunity in the financial services industry by developing impactful Machine Learning solutions As aApplied AI ML Lead within the team at JPMorgan, you will collaborate with all lines of business and functions to deliver software solutions. You will have opportunity to research, experiment, develop, and productionize high-quality machine learning models, services, and platforms to make a significant business impact. You will also design and implement highly scalable and reliable data processing pipelines and perform analysis and insights to promote and optimize business results. Job responsibilities Design, deploy an...

Must have minimum experience of 5 years working on developing data pipelines Must have worked on data engineering using Python for at least 5 years Must have experience working on big data technologies for at least 5 years Must have experience in exploring and proposing solutions and come up with architecture and technical design Must have experience working on Docker Must have experience working on AWS Must have working experience on DevOps pipelines Must be a excellent technical leader who can take responsibility of a team and own it Must have experince practicing agile methods - scrum and kanban Must have experience interacting with US customers on day to day basis

Data Management and Optimization Assure that data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements. Solution Design Solution design using Microsoft Azure services and other tools. Automation and Deployment The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning, etc.). Data Loading and Reporting Structures Load transformed data into storage and reporting structures in destinations including data warehouse, high-speed indexes, real-time reporting systems, and analytics applications. Data Pipeline Development Build data pipelines to collectively bring t...

Data Management and Optimization Assure that data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements. Solution Design Solution design using Microsoft Azure services and other tools. Automation and Deployment The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning, etc.). Data Loading and Reporting Structures Load transformed data into storage and reporting structures in destinations including data warehouse, high-speed indexes, real-time reporting systems, and analytics applications. Data Pipeline Development Build data pipelines to collectively bring t...

Design and implement scalable data processing solutions using Azure Databricks Develop and maintain data pipelines to ingest data from various sources Optimize data storage and retrieval processes for efficiency and performance Ensure data quality and integrity throughout the data lifecycle Troubleshoot and resolve data processing issues Deploy and manage big data solutions in a distributed computing environment Collaborate with data engineers and data scientists to facilitate data accessibility Document data processing solutions and best practices Requirements: Bachelors degree in Computer Science, Information Technology, or a related field 4+ years of experience in data engineering or soft...

Design and implement scalable data processing solutions using Azure Databricks Develop and maintain data pipelines to ingest data from various sources Optimize data storage and retrieval processes for efficiency and performance Ensure data quality and integrity throughout the data lifecycle Troubleshoot and resolve data processing issues Deploy and manage big data solutions in a distributed computing environment Collaborate with data engineers and data scientists to facilitate data accessibility Document data processing solutions and best practices Requirements: Bachelors degree in Computer Science, Information Technology, or a related field 4+ years of experience in data engineering or soft...

Design and implement scalable data processing solutions using Azure Databricks Develop and maintain data pipelines to ingest data from various sources Optimize data storage and retrieval processes for efficiency and performance Ensure data quality and integrity throughout the data lifecycle Troubleshoot and resolve data processing issues Deploy and manage big data solutions in a distributed computing environment Collaborate with data engineers and data scientists to facilitate data accessibility Document data processing solutions and best practices Requirements: Bachelors degree in Computer Science, Information Technology, or a related field 4+ years of experience in data engineering or soft...

Design, develop, and implement large-scale data processing solutions using Java, Spark Streaming, Spark Batch, and Hadoop. Build and maintain data pipelines for real-time and batch processing, ensuring high performance and reliability. Develop and optimize data storage and retrieval systems using Hadoop Distributed File System (HDFS) and related technologies. Utilize AWS services (e.g., EMR, S3, Lambda, Glue, Airflow) to build and deploy cloud-based big data solutions. Implement data ingestion, transformation, and storage processes for various data sources. Optimize Spark applications for performance and scalability. Troubleshoot and resolve complex data processing issues.

We are seeking a highly skilled Senior Data Engineer with 5+ years of experience for our Bengaluru location (max 30 days notice period). The ideal candidate will have strong expertise in designing, developing, and maintaining robust data ingestion frameworks, scalable pipelines, and DBT-based transformations. Responsibilities include building and optimizing DBT models, architecting ELT pipelines with orchestration tools like Airflow/Prefect, integrating workflows with AWS services (S3, Lambda, Glue, RDS), and ensuring performance optimization on platforms like Snowflake, Redshift, and Databricks. The candidate will implement CI/CD best practices for DBT, manage automated deployments, trouble...

Job Description Role & Responsibilities: Develop and maintain SQL Server databases, stored procedures, and SSIS packages. Build and optimize ADF pipelines and integrate data from various sources. Write Python scripts for automation and data handling tasks. Create robust and high-performing ETL solutions in Azure environment. Participate in requirement gathering, debugging, testing, and performance tuning. Collaborate with business and technical teams using Agile practices. Work on API-based data integrations and monitor SQL jobs. Document solutions and maintain coding standards and best practices. Shift Timing: 2:00 PM 11:00 PM IST Work Location: Hyderabad (In-Person) Interview Mode: In-Pers...

Job Title: Product Architecture Planner Platform & Systems Integration Department: Product Strategy / Architecture & Compliance Location: Gurugram Employment Type: Full-Time Experience Required: 25 Years Reporting To: AVP Product & Architecture About the Role As a Product Architecture Planner, you will be responsible for designing, aligning, and ensuring seamless integration of product platforms across multiple ecosystems. You will act as the custodian of product-level architectural standards, compliance protocols, and interoperability guidelines. This role demands an innovative mindset, strong attention to detail, and the ability to draft processes, SoPs, and frameworks that will scale acro...

About Us Anand Rathi Shares and Stock Brokers Limited is a leading financial services company with 30 years of proven market excellence. The company offers a wide range of products and services under brokerage and distribution of equities, commodities, currencies and mutual funds. Anand Rathi has a strong presence in globally and serves a diverse client base. We are scaling our digital-first business with a next-generation, high-performance trading platform that delivers lightning-fast executions and rich market insights to a broader spectrum of our retail clients. Our commitment to transparent, responsible product innovation drives every enhancement: intuitive onboarding flows, seamless mul...

We are seeking a highly skilled Senior Data Engineer with 5+ years of experience for our Bengaluru location (max 30 days notice period) The ideal candidate will have strong expertise in designing, developing, and maintaining robust data ingestion frameworks, scalable pipelines, and DBT-based transformations Responsibilities include building and optimizing DBT models, architecting ELT pipelines with orchestration tools like Airflow/Prefect, integrating workflows with AWS services (S3, Lambda, Glue, RDS), and ensuring performance optimization on platforms like Snowflake, Redshift, and Databricks The candidate will implement CI/CD best practices for DBT, manage automated deployments, troublesho...

We are seeking a highly skilled Data Engineer with 5+ years of experience for our Bengaluru location (max 30 days notice period). The ideal candidate will have strong expertise in designing, developing, and maintaining robust data ingestion frameworks, scalable pipelines, and DBT-based transformations. Responsibilities include building and optimizing DBT models, architecting ELT pipelines with orchestration tools like Airflow/Prefect, integrating workflows with AWS services (S3, Lambda, Glue, RDS), and ensuring performance optimization on platforms like Snowflake, Redshift, and Databricks. The candidate will implement CI/CD best practices for DBT, manage automated deployments, troubleshoot p...

Experience: 3-5 years Experience with Apache Spark is must. Looking for a skilled backend developer with strong experience in Java, Spring Boot, and Apache Spark. Responsible for building scalable microservices and processing large datasets in real-time or batch environments. Must have solid understanding of REST APIs, distributed systems, and data pipelines. Experience with cloud platforms (AWS/GCP) is a plus. Show more Show less

Hi Greetings from Zensar!! We are looking for ETL L2 Support role for Hyderabad location. Exp - 3-6 Years Notice - Immediate to 15 Days 3-5 years of production support experience on Informatica/Python/AWS Technologies and applications. Must have good understanding and technical knowledge on Informatica architecture/client components such as Workflow Manager, Mapping Designer, workflow monitor and Repo manager. Excellent knowledge on AWS/Python concepts. Informatica to Cloud Migration. Hands-on expertise in debugging Informatica ETL Mapping to narrow down the Issue. Hands-on experience in ETL transformation such as lookup/joiners/source qualifier/normalizer. Hands-on experience in dealing wit...

Lexington Partners is one of the worlds largest and most trusted managers of secondary private equity and co-investment funds. Since our founding in 1994, we have been at the forefront of innovation in private equity investing, managing over $70 billion in committed capital and partnering with a global network of institutional investors, private equity firms, and portfolio companies. What are the ongoing responsibilities of Associate Software Engineer (Data Engineer) responsible for We are building a growing Data and AI team. You will play a critical role in the efforts to centralize structured and unstructured data for the firm. We seek a candidate with skills in data modeling, data managem...

Notice Period: Immediate joiners or within 15 days preferred Share Your Resume With: Current CTC Expected CTC Notice Period Preferred Job Location Primary Skills: MSSQL, Redshift, Snowflake T-SQL, LinkSQL, Stored Procedures ETL Pipeline Development Query Optimization & Indexing Schema Design & Partitioning Data Quality, SLAs, Data Refresh Source Control (Git/Bitbucket), CI/CD Data Modeling, Versioning Performance Tuning & Troubleshooting What You Will Do: Design scalable, partitioned schemas for MSSQL, Redshift, and Snowflake. Optimize complex queries, stored procedures, indexing, and performance tuning. Build and maintain robust data pipelines to ensure timely, reliable delivery of data. Ow...

As a leading provider of AI-powered solutions to clean dirty data and unlock its hidden potential for healthcare transformation, HiLabs is committed to revolutionizing the healthcare industry through innovation, collaboration, and a relentless focus on improving patient outcomes. Join a multidisciplinary team of industry leaders, healthcare domain experts, AI/ML, and data science professionals from prestigious institutions worldwide, including Harvard, Yale, Carnegie Mellon, Duke, Georgia Tech, IIM, and IIT. Be part of a team that leverages advanced AI, ML, and big data technologies to develop cutting-edge healthcare technology platforms and deliver innovative business solutions. Position: D...

You are looking for a Senior Delivery Manager to lead a cross-functional team in implementing and configuring product solutions for enterprise-level clients. This role requires strong technical leadership, people management, and project ownership skills, ensuring quality checks and innovative approaches to streamline delivery. Collaboration with internal engineering, R&D, QA, and support teams is crucial, with high-level requirements coming from Technical Program Managers. As a Senior Delivery Manager, you will be responsible for: Strategic Leadership & Team Management: - Managing, mentoring, and inspiring a team of engineers and project leads for high performance, engagement, and growth - D...

SAIGroup is a private investment firm that has committed $1 billion to incubate and scale revolutionary AI-powered enterprise software application companies. Our portfolio consists of rapidly growing AI companies catering to over 2,000+ major global customers, generating almost $800 million in annual revenue, and employing a global workforce of over 4,000 individuals. We invest in new ventures based on breakthrough AI-based products that have the potential to disrupt existing enterprise software markets. Our latest investment, JazzX AI, is a pioneering technology company focused on shaping the future of work through an AGI platform designed for the enterprise. JazzX AI is not just creating a...

You will be responsible for assembling large, complex sets of data that meet non-functional and functional business requirements. You will develop and maintain scalable data pipelines, as well as build new API integrations to support increasing data volume and complexity. Collaboration with analytics and business teams is crucial to improve data models feeding business intelligence tools, increasing data accessibility, and promoting data-driven decision making across the organization. Your role will involve building the necessary infrastructure for optimal extraction, transformation, and loading of data from various sources using SQL and other technologies. You will implement processes and s...