1175 Data Pipeline Jobs - Page 21

Job Purpose This role is central to managing the airlines operational data, building scalable data pipelines, enabling organization-wide reporting, and driving automation using the Microsoft ecosystem. Key Accountabilities Functional Activities Leading a team of experienced data engineers and analyst Design and maintain end-to-end data pipelines using SQL, Microsoft Fabric, and Python to support operations and planning analytics. Develop and manage centralized databases and reusable data models for seamless integration across Power BI, Power Apps, and other analytics platforms. Build and optimize Power BI dashboards and data models, implementing Row-Level Security (RLS), performance tuning, ...

We have urgent requirement for Data Engineer position. Must be able to join in 15 to 20 days . Must relocate to bangalore if shortlisted. Required skills- Must have experience in projects related to banking domain & loans and NPA or regulatory reporting. Strong experience in Oracle SQL / PL-SQL. Proficient in PySpark for handling big data processing. Deep understanding of data warehouse concepts (star/snowflake schema, facts & dimensions, SCDs etc.)

Roles and Responsibilities Design, develop, test, deploy, and maintain large-scale data pipelines using Snowflake as the primary database engine. Collaborate with cross-functional teams to gather requirements and design solutions that meet business needs. Develop complex SQL queries to extract insights from large datasets stored in Snowflake tables. Troubleshoot issues related to data quality, performance tuning, and security compliance. Participate in code reviews to ensure adherence to coding standards and best practices. Desired Candidate Profile 3-7 years of experience working with Snowflake as a Data Engineer or similar role. Strong understanding of SQL programming language with ability...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

The Natural Language Query (NLQ) platform is an innovative initiative designed to transform the way users interact with data. Our platform leverages advanced natural language processing (NLP) to convert user queries in plain language into executable SQL queries, enabling seamless data retrieval and analysis without the need for SQL expertise. The NLQ platform will be powered by the Enterprise Data Catalog, ensuring comprehensive and accurate metadata definitions for all table and column information. This platform empowers different business units to build external-facing conversational BI chatbots to handle customer requests, while also significantly reducing data exploration efforts for int...

Greetings from Global Healthcare Billing Partners Private Limited..! Job Title: Python Developer - Desktop Application Location: Velachery, Chennai Experience: 1 to 3 Years About the Role: We are seeking a highly skilled Python Desktop Developer with strong expertise in Desktop Application Development . The ideal candidate should have hands on experience in building scalable applications, writing efficient code, and collaborating with cross functional teams to deliver innovative solutions for the healthcare domain. Key Responsibilities: Good Knowledge & expertise Python: Strong expertise in Core Python with clean, efficient coding practices. Hands-on experience in Desktop application framewo...

If you are a smart & passionate team player - then this Senior Data Engineer [Snowflake] opportunity is for you! We at IMRIEL(An Allata Company) are looking for a Senior Data Engineer to implement methods to improve data reliability and quality. You will combine raw information from different sources to create consistent and machine-readable formats. You will also develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling. Resourcefulness is a necessary skill in this role. If you truly love gaining new technical knowledge and can add more awesomeness to the team, you are eligible! Experience : 5 to 8 years. Location : Vadodara & Pune...

Salary: 25 to 40 LPA Exp: 5 to 10 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Da...

Job Title - Snowflake Data Engineer Key Skills - Snowflake , Snowpipe , Data Warehousing , Advance SQL Job Description: Design, develop, and maintain robust ELT/ETL data pipelines to load structured and semi-structured data into Snowflake. Implement data ingestion workflows using tools like Azure Data Factory, Informatica, DBT, or custom Python/SQL scripts. Write and optimize complex SQL queries, stored procedures, views, and UDFs within Snowflake. Use Snowpipe for continuous data ingestion and manage tasks, streams, and file formats for near real-time processing Optimize query performance using techniques like clustering keys, result caching, materialized views, and pruning strategies. Moni...

The Data Engineer should have strong data infrastructure and data architecture skills, a proven track record of administration for data engineering teams, strong operational skills to drive efficiency and speed, strong project management skills, and a core value base for how analytics can proactively improve organizations. You will be part of the Data & Analytics team and will partner closely with a team of data scientists, business analysts & business intelligence leading a cloud-based strategy for our clients. You'll work on our Azure based data pipeline projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights us...

Only immediate joiner preferred. Experience: 8-12 years Responsibilities: Design, develop & maintain ETL pipelines using Python, PySpark & SQL on AWS/Azure/GCP. Collaborate with cross-functional teams for data transformation & quality assurance.

We are seeking Data Engineers (GCP) who can develop and maintain a high-performance streaming data platform for a leading healthcare CRM product. You will work across technologies like Python, PySpark, SQL, Continuous Integration/ Continuous Development, Kafka, GCP - BigQuery, Google cloud Compute, Looker, Data Modelling. Required Skill Set: Programming & Data Engineering: Python, PySpark, SQL, CICD Streaming & Orchestration: Kafka, GCP Composer/Airflow Cloud Tools: Google Cloud Platform (Dataproc, BigQuery, Looker) Data Architecture: Data Warehousing, Data Modeling Location : Manyata Tech Park, Bengaluru Roles and Responsibilities Responsibilities: Design, develop, and maintain scalable dat...

Key Responsibilities: Design, develop, and maintain scalable and secure ETL pipelines using PySpark , Python , and SQL . Build and orchestrate data workflows leveraging AWS services such as Glue , Lambda , Step Functions , S3 , Athena , and Redshift . Perform data profiling, cleansing, and transformation to ensure data quality and consistency. Optimize and tune data pipelines for performance and cost-efficiency. Work with data architects, analysts, and stakeholders to understand data requirements and translate them into technical solutions. Develop and maintain technical documentation for all ETL processes and data flows. Monitor data pipelines and troubleshoot any issues related to performa...

World Wide Technology Holding Co, LLC (WWT) is a privately held global technology integrator and supply chain solutions provider. Through our culture of innovation, we inspire, build and deliver business results, from idea to outcome. Based in St. Louis, WWT works closely with industry leaders such as Cisco, VMware, NetApp, Dell EMC, HPE and several others, but also include emerging tech players. WWT employs more than 10,000 people and operates in more than 2 million square feet of state-of-the-art warehousing, distribution and integration space strategically located throughout the world. WWT ranks Glassdoors Best Places to Work and has been on Fortune’s 100 Best Companies to Work For list 1...

At Tredence AWS Data Engineer designs, builds, and maintains scalable data pipelines and infrastructure using AWS services like Glue, S3, and Redshift, focusing on data ingestion, transformation (ETL), modeling, and quality. They write SQL and Python code, orchestrate workflows with tools like Airflow, and collaborate with data scientists and analysts, aiming to ensure data is accurate, accessible, and optimized for analysis and business decision-making. Role - AWS Data Engineer Years of Experience: 2~5 years Roles and responsibilities The day-to-day work of a junior-to-mid AWS data engineer involves a mix of development, monitoring, and collaboration. Pipeline development: Designing, buildi...

Design and maintain high-performance SQL queries to support analytics and real-time decision-making. Build and scale distributed data pipelines to support a variety of e-commerce functions including order tracking, recommendations, and dynamic pricing. Implement real-time data processing using RisingWave, Azure , or ect delivering insights on user behavior, product trends, and operational efficiency. Lead and mentor a team of data engineers, setting technical direction and ensuring high-quality delivery. Collaborate with product, engineering, and analytics teams to translate business needs into scalable data solutions. Own and drive data-driven initiatives across the organization with measur...

The GCP Data Engineer will be responsible for designing, developing, and maintaining data pipelines and data infrastructure on Google Cloud Platform (GCP). This role requires expertise in data engineering best practices, cloud architecture, and big data technologies. The ideal candidate will work closely with data scientists, analysts, and other stakeholders to ensure the availability, reliability, and efficiency of data systems, enabling data-driven decision-making across the organization. Key Responsibilities Data Pipeline Development Design, develop, and maintain scalable and efficient ETL/ELT pipelines on GCP. Implement data ingestion processes from various data sources (e.g., APIs, data...

Job Description: We are seeking talented and motivated Data Engineers to join our dynamic team and contribute to our mission of harnessing the power of data to drive growth and success. As a Data Engineer at VXI Global Solutions, you will play a critical role in designing, implementing, and maintaining our data infrastructure to support our customer experience and management initiatives. You will collaborate with cross-functional teams to understand business requirements, architect scalable data solutions, and ensure data quality and integrity. This is an exciting opportunity to work with cutting-edge technologies and shape the future of data-driven decision-making at VXI Global Solutions. R...

Role & responsibilities 4+ years in Data warehousing/ Business Intelligence field Architecture design and implementation experience with medium to complex on-prem to cloud migrations with any of the major cloud platforms (preferably Azure). Minimum 3+ years experience in Azure database offerings [Relational, NoSQL, Datawarehouse. 2+ years hands-on experience in various Azure services preferred Azure Data Factory, Kafka, Azure Data Explorer, Storage, Azure Data Lake, Azure Synapse Analytics, Azure Analysis Services & Databricks. Minimum of 3 years of hands-on database design, modelling and integration experience with relational data sources, such as SQL Server databases, Oracle/MySQL, Azure S...

About the Company Agoda is an online travel booking platform that offers accommodations, flights, and more. With a global network of 4.7M hotels and holiday properties, as well as flights, activities, and other travel options, Agoda connects travelers worldwide. As part of Booking Holdings and based in Asia, our diverse team of 7,100+ employees from 95+ nationalities across 27 markets fosters a work environment rich in diversity, creativity, and collaboration. We embrace a culture of experimentation and ownership to enhance our customers" travel experiences. Purpose At Agoda, we believe in bridging the world through travel. Travel not only allows people to enjoy and experience the world but ...

We are looking for a skilled and experienced Databricks Engineer to join our team at our Hyderabad (Madhapur) office. The ideal candidate will have a strong background in data engineering, hands-on experience with Databricks, and a solid understanding of cloud platforms (AWS, Azure, or GCP). This is a great opportunity to work on cutting-edge data projects in a collaborative and fast-paced environment. Key Responsibilities : - Design and implement scalable and efficient data pipelines, ETL/ELT processes, and data integration solutions using Databricks. - Work with large-scale structured and unstructured datasets using Python, PySpark, SQL, and NoSQL technologies. - Collaborate with cross-fun...

Job Description AI/ML Engineer (5 Years Experience) About Volante: Volante is a leading Financial Services technology company providing a market-leading, cloud-native Payments Processing Platform to banks and financial institutions globally. By joining our product team, you will have an opportunity to shape the future of payments technology while working on cutting-edge AI/ML solutions. Role Overview: We are seeking an experienced AI/ML Engineer with ~5 years of hands-on experience in building, deploying, and scaling machine learning solutions. The ideal candidate will have strong expertise in Python, deep learning, generative AI, and end-to-end model deployment (cloud and on-premises), with...

Why Ness We know that people are our greatest asset. Our staff’s professionalism, innovation, teamwork, and dedication to excellence have helped us become one of the world’s leading technology companies. It is these qualities that are vital to our continued success. As a Ness employee, you will be working on products and platforms for some of the most innovative software companies in the world. You’ll gain knowledge working alongside other highly skilled professionals that will help accelerate your career progression. You’ll also benefit from an array of advantages like access to trainings and certifications, bonuses, and aids, socializing activities and attractive compensation. Key Responsi...

Industry/Domain: Product Engineering Big Data, Healthcare, Cyber Security, Mobile. Few words about Unify Technologies: Unify a Digital Engineering company. We help our clients by providing Digital Engineering Services to develop high-quality products. We have extensive experience in software product engineering and a successful track record of delivering on aggressive delivery plans without compromising on the quality in Cloud, Mobile, and Data practices. Job Description: Experience : 4 to 7 Year Notice Period: Immediate joiners / able to join within 15Days. The role involves working with a FAANG Company's Audio Research Lab to develop robust data tooling and infrastructure. This includes im...