3201 Data Pipelines Jobs - Page 30

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Graph Databases Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating with Integration Architects an...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark, Microsoft SQL Server, Python (Programming Language), Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and proc...

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Graph Databases Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating with Integration Architects an...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Python (Programming Language), Microsoft SQL Server, Databricks Unified Data Analytics Platform, PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processin...

About The Role Project Role : Responsible AI Tech Lead Project Role Description : Ensure the ethical and responsible use of artificial intelligence (AI) technologies. Design and deploy Responsible AI solutions; align AI projects with ethical principles and regulatory requirements. Provide leadership, fosters cross-functional collaboration, and advocates for ethical AI adoption. Must have skills : Generative AI Good to have skills : Python (Programming Language), 3D Designing, Machine Learning Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :We are seeking an experienced AI Team Lead with strong architectural and backend engineeri...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Google Cloud Platform Architecture Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years of fulltime education Role Summary :The Junior GCP Data Engineer (Ingestion) contributes to data pipeline development and ingestion processes within the Google Cloud Platform (GCP) ecosystem. This role involves in building, testing...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Microsoft Power Business Intelligence (BI) Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensu...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark, Adobe Suite Good to have skills : MySQL, Python (Programming Language), AWS BigData, Service design exposure Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :We are looking for a Data Engineer skilled in Python, PySpark, SQL, AWS Glue, Redshift, and Airflow to build and maintain scalable data pipeline...

Role Overview We are seeking an experienced GCP Data Engineer with strong expertise in designing, developing, and managing large-scale data pipelines on Google Cloud Platform. The ideal candidate will have hands-on experience in Big Data technologies (Hadoop, PySpark, BigQuery) and will collaborate closely with cross-functional teams and clients to deliver high-quality data solutions. Key Responsibilities Design, build, and maintain data pipelines and ETL workflows on GCP. Develop and optimize PySpark scripts and Spark-based data processing solutions. Work with BigQuery to manage datasets, write complex SQL queries, and optimize data performance. Utilize the Hadoop ecosystem (HDFS, Hive, Map...

Key Responsibility Area: ( Specifies Key Result Areas for the Incumbent ) o Engage with clients to precisely identify and fulfill their data engineering needs. o Lead and manage special projects to meet strategic goals. o Develop advanced models with Snowflake and SSIS to enhance data accuracy and utility. o Continuously refine data processing rules and procedures for optimal results. o Design and implement scalable data architectures using Snowflake. o Maintain and enhance data pipelines, integrating new data sources and APIs as needed. o Monitor and ensure high data quality across systems for reliable decision-making. o Utilize SSIS for efficient data extraction, transformation, and loadin...

US Shift Key Responsibility Area: ( Specifies Key Result Areas for the Incumbent ) o Engage with clients to precisely identify and fulfill their data engineering needs. o Lead and manage special projects to meet strategic goals. o Develop advanced models with Snowflake and SSIS to enhance data accuracy and utility. o Continuously refine data processing rules and procedures for optimal results. o Design and implement scalable data architectures using Snowflake. o Maintain and enhance data pipelines, integrating new data sources and APIs as needed. o Monitor and ensure high data quality across systems for reliable decision-making. o Utilize SSIS for efficient data extraction, transformation, a...

Data Architecture Design and Implementation: Design and implement cloud-based data architectures leveraging Databricks, AWS, Azure, or GCP. Architect end-to-end data pipelines that integrate with Databricks and other cloud services. Ensure that the data architecture supports both batch and real-time data processing and analytics.

IT development experience with minimum 3+ years hands-on experience in Snowflake. Strong experience in building/designing data warehouses or data lakes, and data mart end-to-end implementation, focusing on large enterprise scale and Snowflake implementations on any of the hyper scalers. Strong experience with building production-ready data ingestion and data pipelines in Snowflake. Good knowledge of Snowflake's architecture and features such as Zero-Copy Cloning, Time Travel, and performance tuning capabilities. Experience with Snowflake RBAC and data security. Strong experience in various Snowflake features, including new features. Experience in Python/PySpark. Experience with AWS services ...

Strong Python programming skills. Experience in workflow orchestration using Apache Airflow. Experience in productionizing machine learning models in Domino run-time environment. Experience in building data pipelines using Python and PySpark. Strong SQL skills and experience in extracting data from different databases (Hive, SnowFlake, etc.), performing data transformation, etc. Preferable experience working with Hadoop ecosystem - HDFS, Hive. Experience working with a job scheduler, preferably Control-M. Experience with Unix-based command line interface and Bash scripts. Experience in developing and querying APIs - Restful, OpenAPI, etc. Strong understanding of DevOps and CI/CD principles; ...

Experience: 5+ years as STRIIM Developer Domain: Telecom domain experience is a plus Communication: Excellent verbal and written communication skills Responsibilities: Build and configure real-time data pipelines Work on CDC connectors and transformations Integrate Striim with data lakes and downstream systems Qualifications Bachelor of Engineering

Collaborate with Palantir FDEs to design and build data pipelines, workflows, and applications in Palantir Foundry and AIP. Integrate diverse enterprise data sources (structured, semi-structured, and unstructured) into Palantirs ontology framework. Develop scalable APIs, microservices, and application components to support business processes and analytics. Design and implement security, compliance, and governance controls aligned with enterprise policies. Qualifications: Bachelor of Technology

Must have 4+ years of IT experience, with at least 2 years of relevant experience in Snowflake. In-depth understanding of Data Warehousing, ETL concepts, and modeling structure principles. Experience working with Snowflake functions, hands-on experience with Snowflake utilities, stage and file upload features, time travel, fail safe, procedure writing, tasks, Snowpipe, and SnowSQL. Knowledge of Snowflake Architecture. Good knowledge of Advanced SQL with joins, views/materialized views, various functions, dimensional data modeling, and ETL procedures using SQL. Experience in Python functional programming is a must, with Pandas, Apache Airflow, etc. Basics of ML & Gen AI. Expertise in engineer...

Build, monitor, and schedule Azure Data Factory (ADF) pipelines. Develop Power BI reports and metrics with strong experience in developing DAX queries. Proficient in SQL queries, including filters, joins, group by, sub-queries, indexing, and optimization. Design, develop, and maintain data pipelines and machine learning models using Azure services such as Azure Data Factory, Databricks, Synapse Analytics, and Azure Machine Learning. Build and optimize data models for analytics and reporting, ensuring high data quality and performance. Collaborate with business stakeholders to gather requirements, create dashboards, and deliver actionable insights using Power BI or similar tools. Ensure secur...

We are looking for a skilled AWS Data Engineer with 6 to 8 years of experience in the field. The ideal candidate will have expertise in designing and implementing data pipelines using AWS services. Roles and Responsibility Design, develop, and maintain large-scale data pipelines using AWS services such as S3, Lambda, and EMR. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and implement data quality checks and validation processes to ensure data integrity. Optimize data processing workflows for performance, scalability, and reliability. Troubleshoot and resolve complex technical issues related to data engineering projects. Participate in code ...

We are looking for a skilled Java/Spark, Scala professional with 5 to 10 years of experience. The job location is flexible. Roles and Responsibility Design, develop, and implement scalable data processing systems using Java/Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data pipelines and architectures. Troubleshoot and optimize system performance issues. Participate in code reviews and ensure high-quality code standards. Stay updated with industry trends and emerging technologies in Java/Spark. Job Requirements Strong proficiency in Java/Spark and Scala programming languages. Experience with big data processin...

Building and optimizing data infrastructure. Enabling scalable data pipelines. Ensuring efficient data processing using AWS and Databricks. Qualification Bachelor of Computer Application

Design, develop, and deploy end-to-end data engineering solutions on the Databricks platform. Build scalable, reliable, and optimized data pipelines that support large-scale analytics, machine learning, and real-time data processing. Utilize Databricks to integrate, transform, and analyze big data from various sources. Desired Skills AWS, Big Data, Spark, Apache, Data Modeling Qualifications Bachelor of Engineering

Desired Skills AWS Data Modelling Python Key Technical Skills & Responsibilities Design, develop, and implement ETL processes to extract, transform, and load data into Snowflake. Utilize DBT for efficient and scalable data transformations to create structured JSON, ensuring data quality and integrity. Implement and manage slowly changing dimensions to support evolving data requirements. Collaborate with cross-functional teams to understand data needs and design solutions that align with business objectives. Optimize and fine-tune data pipelines for performance and scalability. Troubleshoot and resolve data-related issues, ensuring the reliability of the data infrastructure. Stay updated on i...

Responsibilities & Requirements: Strong hands-on experience in Python programming and PySpark. Experience using AWS services such as RedShift, Glue, EMR, S3, and Lambda. Experience working with Apache Spark and the Hadoop ecosystem. Must have Data Warehouse experience with AWS Redshift or Hive. Experience in implementing security measures for data protection. Expertise in building and testing complex data pipelines for ETL processes (batch and near real time). Qualifications Bachelor of Technology

Good coding skills in Python and experience with machine learning (Classical / GenAI) , as well as large language models (LLMs) usage . Hands-on experience with MLOps/LLMOps for model training, deployment, and monitoring. Knowledge of Docker/Kubernetes and at least one major cloud platform (AWS/Azure/GCP). Ability to build data pipelines and ensure data security and compliance . Experience setting up dashboards to track model health, drift, and performance. Desired Skills AWS | Azure | Python Qualifications Bachelor of Engineering