1569 Data Pipeline Jobs - Page 7

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description : Industry & Sector : Technology consulting firm operating in the blockchain, fintech and distributed ledger technology (DLT) space. We deliver enterprise-grade DLT integrations, ledger analytics and data engineering solutions for global clients, transforming on-chain and off-chain data into actionable business insights. Role : Senior Data Engineer (DLT) Remote (India). Role & Responsibilities : - Design and build scalable ETL/ELT pipelines to ingest, process and transform on-chain and off-chain ledger data using streaming and batch frameworks. - Integrate with DLT nodes, RPC endpoints and indexers to capture transactions, blocks and smart-contract events into analytical stores. ...

Description: Job Description We are seeking an experienced Senior Data Scientist with strong expertise in Generative AI to design, develop, and deploy production-grade machine learning solutions. This role combines data science, data engineering, and cloud-based deployment, with a focus on building reliable models in a regulated operational environment. Requirements: 6+ years of experience in Data Science or Advanced Analytics Strong expertise in Generative AI models and techniques Proven data engineering experience with production pipelines Proficiency in Python and data science libraries Experience with Spark and AWS or Azure Strong communication and collaboration skills Aviation domain ex...

As a Senior Data Pipeline Engineer at Agoda, your role will involve working on distributed systems across multiple data centers, managing thousands of servers and processing hundreds of billions of messages daily. Your responsibilities will include ensuring data quality, integrity, and accuracy, designing scalable systems to handle increasing data volume, and managing projects with a small team to enhance your leadership skills. You will also play a key role in building, administering, and scaling data pipelines, developing frameworks used company-wide, managing data ingestion into various systems, and monitoring data accuracy SLAs. Key Responsibilities: - Build, administer, and scale data p...

As an Oracle Fusion Data Intelligence (FDI) developer, your role will involve enhancing the reporting capabilities across the R13 HCM and Financials suite of products. Your expertise with FDI/ADW Oracle analytics cloud and developing customized dashboards will be crucial for supporting ongoing development and larger initiatives within the business application environment. Your responsibilities will include: - Providing workstream leads with an understanding of the standard KPIs and metrics delivered by HCM, SCM, ERP analytics. - Developing customized dashboards covering HCM, SCM, ERP when standard metrics do not meet business requirements. - Proficiency with out-of-the-box semantic model for...

As a senior data pipeline engineer at Agoda, you will be working on distributed systems that span multiple data centers, thousands of servers, and process hundreds of billions of messages a day. Your role will focus on ensuring data quality, integrity, and accuracy while designing new scalable systems to handle the increasing data volume. You will have the opportunity to manage projects with a small team, enhancing your ownership and leadership skills. Your responsibilities will include building, administering, and scaling data pipelines, developing and expanding existing frameworks used by teams, managing data ingestion into multiple systems, building tools to monitor data accuracy SLAs, ex...

Role & responsibilities Required Skillset : Technical Lead - 7+yrs ( Databricks, GCP) Strong Hands-on experience : Databricks PySpark SQL Terraform CI-CD GCP Services - Cloud build, BigQuery, Firestore etc Airbyte (Intermediate level) Datadog (Intermediate level

Salary: 15 to 30 LPA Exp: 3 to 8 years Location : Gurgaon/Bangalore/Pune/Chennai Notice: Immediate to 30 days..!! Key Responsibilities & Skillsets: Common Skillsets : 3+ years of experience in analytics, Pyspark, Python, Spark, SQL and associated data engineering jobs. Must have experience with managing and transforming big data sets using pyspark, spark-scala, Numpy pandas Excellent communication & presentation skills Experience in managing Python codes and collaborating with customer on model evolution Good knowledge of data base management and Hadoop/Spark, SQL, HIVE, Python (expertise). Superior analytical and problem solving skills Should be able to work on a problem independently and p...

Salary: 35 to 50 LPA Exp: 7 to 12 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Da...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Design and maintain scalable ETL/ELT data pipelines Validate, enrich & prepare data for analytics & ML Manage schemas, versioning & data contracts Use PostgreSQL/SQLite, Spark/DuckDB & Airflow At dext ai, we refer candidates to AI-training companies Required Candidate profile Background in CS, data engineering or information systems Strong in Python, pandas, SQL Experience with PostgreSQL/SQLite Knowledge of Spark or DuckDB Workflow orchestration with Airflow Perks and benefits WFH at $14 per hour with flexible schedule

Experienced Developer Azure services, including developing, deploying, and configuring Azure Web Apps/App Services, cloud migration and integration exposure. Quick learner with feedback and proven experience developing client project environments. Required Candidate profile Core: Azure Modern Data Platform, SQL Server (T-SQL) Power BI, Synapse, Azure Data Bricks Supporting: Azure ML, Azure infrastructure, Python, PowerShell, .Net, Data Factory, Azure Analytics Services

--Data Engineer --GCP --Terraform --Big Query, Data Flow, Pub/sub, Cloud Functions etc. -Scrum development practices, using associated tools. e.g. JIRA, Confluence -Experience with design, development and testing of Data Pipelines. SQL and Python

Location City Mumbai Department Intelligent Automation and Accelerated Analytics Experience 1 - 2 Years Design, develop, and maintain scalable data pipelines to support ingestion, transformation, and processing of large structured and unstructured datasets using modern data engineering frameworks. Collaborate with cross-functional teams to understand data requirements, ensure data quality, and optimize data models for analytics, reporting, and machine learning use cases. Implement efficient ETL/ELT workflows, monitor performance, troubleshoot issues, and enhance pipeline reliability while adhering to best practices in data governance and security. Automate data workflows, validate datasets, ...

We are looking for a highly experienced Senior Machine Learning Engineer with 10+ years of combined experience in software engineering and machine learning. This role requires end-to-end ownership of scalable ML model developmentspanning data preparation, feature engineering, model training, deployment, and optimization in production. The ideal candidate excels at building robust ML systems, collaborating with cross-functional teams, and delivering high-performance machine learning solutions at scale. Key Responsibilities Design, build, and deploy machine learning models along with complete end-to-end ML pipelines. Engineer high-quality features and manage large, complex datasets for model t...

SUMMARY Job Title: Data Analyst Job Location: Hyderabad Start Date: As soon as possible Key Responsibilities: Collaborate with cross-functional teams to understand data requirements and objectives. Design and implement robust data pipelines for efficient data extraction, loading, and transformation (ELT) processes. Analyze large datasets by using statistical techniques to identify trends, patterns, and insights relevant to business goals. Clean and transform data to ensure accuracy and consistency. Identify, understand the reason behind it, and take actions to resolve data quality issues. Develop and maintain reports and dashboards to communicate findings to stakeholders. Provide data-driven...

Job Title: Databricks Developer Job Description:A Pyspark and Databricks Developer with a good understanding of the entire ETL/Azure lifecycle with a background of data projects. Responsibilities Design, develop, and maintain scalable data pipelines and ETL processes using Azure Databricks, Data Factory, and other Azure services Implement and optimize Spark jobs, data transformations, and data processing workflows, Managing Databricks notebooks, Delta lake with Python, Delta Lake with Sparks SQL in Databricks Leverage Azure DevOps and CI/CD best practices to automate the deployment /DAB Deployments and management of data pipelines and infrastructure Ensure Data Integrity checks and Data Qual...

Role & responsibilities Team Player and Self-Starter, experienced in : Apache Airflow, Apache Spark, Snowflake, SQL, ETL concepts, data quality assurance Build and manage end-to-end data workflows using Apache Airflow and Spark. Design ETL processes to support a variety of analytics and business needs. Oversee data validation and quality controls leveraging SQL and Snowflake. Optimize pipeline performance for speed and reliability.

Role Overview: We are looking for a skilled Database & Data Engineer to design, build, and maintain robust data solutions on AWS . The ideal candidate will have hands-on experience with RDS (Oracle & PostgreSQL) , data lakes , and streaming platforms , ensuring efficient data flow and high availability across systems. Key Responsibilities: Develop and maintain data pipelines using Apache NiFi and Spark for batch and streaming data processing. Manage AWS RDS instances for Oracle and PostgreSQL , including configuration, performance tuning, and troubleshooting. Implement and optimize data lakes on AWS S3 using Apache Iceberg for structured and semi-structured data. Build and manage real-time d...

Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be skilled in ETL tools such as Informatica Glue Databricks and DataProc with coding expertise in Python PySpark and SQL. Works independently and has a deep understanding of data warehousing solutions including Snowflake BigQuery Lakehouse and Delta Lake. Capable of calculating costs and understanding performance issues related to data solutions. Outcomes: Act creatively to develop pipelines and applications by selecting appropriate technical options optimizing application development mainten...