6525 Data Engineering Jobs - Page 29

What You Will Do : Design and lead end-to-end architecture for modern, cloud-native data engineering, AI/ML and analytics platforms across the full data lifecycle, including ingestion, storage, transformation, and consumption. Architect high-performance data solutions using Azure Databricks, Fabric, Snowflake, AWS Redshift, Power BI, Tableau, Python, and other relevant technologies. Collaborate with technology leadership and engineering teams to align solutions with enterprise strategy, business goals, and innovation roadmaps. Define and enforce standards for data quality, metadata, lineage, and governance in partnership other teams using market leading tools. Provide architectural guidance ...

Candidate will have a strong foundation in Python (including libraries like pandas, numpy, folium, scikit-learn and multi-processing/threading), MySQL, BigQuery. Solid understanding of data warehousing, ETL frameworks, & cloud-based data platforms.

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way youd like, where youll be supported and inspired bya collaborative community of colleagues around the world, and where youll be able to reimagine whats possible. Join us and help the worlds leading organizationsunlock the value of technology and build a more sustainable, more inclusive world. Job Role Very good Understanding of current work and the tools and technologies being used. Comprehensive knowledge and clarity on Bigquery, ETL, GCS, Airflow/Composer, SQL, Python. Experience working with Fact and Dimension tables, SCD. Minimum 3 years" experience in GCP Data Engineering. Java/ Pytho...

About the Role : We are seeking a highly experienced and proactive SAP Cloud Data Architect to join our team for an urgent 12-month contract. This is a critical role for an individual who thrives in a fast-paced environment and can hit the ground running. You will be instrumental in leading complex data integration initiatives and optimizing our SAP data landscape. Key Responsibilities : - SAP ECC/BW/HANA Solution/Data Architecture : Design and implement end-to-end SAP ECC, BW, and HANA data architectures, ensuring the delivery of scalable, robust, and high-performance solutions. - Data Model & ETL Optimization : Develop, optimize, and maintain data models, ETL processes, and reporting frame...

About the Job : We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. In this pivotal role, you will be instrumental in driving our data engineering initiatives, with a strong emphasis on leveraging Dataiku's capabilities to enhance data processing and analytics. You will be responsible for designing, developing, and optimizing robust data pipelines, ensuring seamless integration of diverse data sources, and maintaining high data quality and accessibility to support our business intelligence and advanced analytics projects. This role requires a unique blend of expertise in traditional data engineering principles, advanced data modeling, and a forward-...

Role : Senior MarkIT EDM Data Engineer We are seeking a highly skilled and motivated Senior MarkIT EDM Data Engineer to join our team. The ideal candidate will be a self-starter who can work independently and collaborate effectively with both internal and external stakeholders. This role requires a deep understanding of data engineering principles, strong technical skills, and the ability to drive projects to successful completion. Key Responsibilities : - Design, develop, and maintain data solutions using MarkIT EDM. - Collaborate with internal teams and external stakeholders to gather requirements and deliver data solutions. - Ensure data quality, integrity, and security across all data pr...

We are looking for a skilled Data Engineer with strong hands-on experience in Clickhouse, Kubernetes, SQL, Python, and FastAPI, along with a good understanding of PostgreSQL. The ideal candidate will be responsible for building and maintaining efficient data pipelines, optimizing query performance, and developing APIs to support scalable data services. - Design, build, and maintain scalable and efficient data pipelines and ETL processes. - Develop and optimize Clickhouse databases for high-performance analytics. - Create RESTful APIs using FastAPI to expose data services. - Work with Kubernetes for container orchestration and deployment of data services. - Write complex SQL queries to extrac...

Job Description: This position is on the Connectivity Enabled Solutions Team in Connectivity Department within the CAT Digital organization. This team has accountability for Building common datasets to support Connectivity initiatives. Streamline reporting of data through various sources, enhancing business logic to improve data reporting, and delivering visualize data in a consistent way for internal and external business users. Building solutions for CAT and dealers to better support the customers We are seeking a skilled and motivated Data engineer with 3-7 years of experience to join our team. The resource should have hands-on experience in SQL development, Snowflake data warehousing, an...

Skills: 1] Hands-on experience in LLM/ML based application development 2] Productionizing LLM/ML apps Data Handling/processing/engineering Dataset creation/curation 3] Understanding of different LLM performance metrics, fine-tuning, prompt engineering 4] Image/Video Processing 5] Generative AI, OpenAI, Claude Knowledge. Good in Prompt Engineering, autogen or similar agentic framework knowledge Skills : - LLM/ML ,application development, Data Handling,Data processing,Data engineering,Dataset creation, LLM performance metrics, fine-tuning, prompt engineering, Image/Video Processing, Generative AI, OpenAI, Claude, Prompt Engineering, autogen, agentic framework Mandatory Key Skills application d...

Required Skills & Experience Data Engineering & ETL7+ years in data architecture, pipelines, or ETL frameworks (Airflow, dbt, Spark, Kafka, etc.). Proven experience building pipelines that integrate multiple external APIs. Strong skills in data validation, mapping, and schema design.Search Engines & Marketplace TechHands-on experience with Typesense, Elasticsearch, or Solr. Background in marketplace or price aggregation systems (e.g., e-commerce, travel, fintech, or cloud pricing). AI & NLP Experience with NLP / semantic search (e.g., vector embeddings, Pinecone, Weaviate, FAISS, OpenAI/LLM APIs). Knowledge of relevance ranking, recommendation systems, and personalization. Programming & Clou...

We are looking for a skilled Data Engineer with strong hands-on experience in Clickhouse, Kubernetes, SQL, Python, and FastAPI, along with a good understanding of PostgreSQL. The ideal candidate will be responsible for building and maintaining efficient data pipelines, optimizing query performance, and developing APIs to support scalable data services. - Design, build, and maintain scalable and efficient data pipelines and ETL processes. - Develop and optimize Clickhouse databases for high-performance analytics. - Create RESTful APIs using FastAPI to expose data services. - Work with Kubernetes for container orchestration and deployment of data services. - Write complex SQL queries to extrac...

About The Opportunity : Join a pioneering consulting firm in the Data Analytics and Cloud Solutions sector, where transformative data architectures empower global enterprises. We specialize in leveraging cutting-edge Snowflake technologies and innovative cloud solutions to drive real-time insights and business intelligence. This remote role, based in India, offers the opportunity to work on high-impact projects while collaborating with a diverse team of experts. Role & Responsibilities : - Design, implement, and optimize scalable data warehousing solutions using Snowflake Cortex. - Develop robust ETL pipelines that ensure data quality, reliability, and efficient integration across platforms....

About the job : Role : Microsoft Fabric Data Engineer Experience : 6+ years as Azure Data Engineer including at least 1 E2E Implementation in Microsoft Fabric. Responsibilities : - Lead the design and implementation of Microsoft Fabric-centric data platforms and data warehouses. - Develop and optimize ETL/ELT processes within the Microsoft Azure ecosystem, effectively utilizing relevant Fabric solutions. - Ensure data integrity, quality, and governance throughout Microsoft Fabric environment. - Collaborate with stakeholders to translate business needs into actionable data solutions. - Troubleshoot and optimize existing Fabric implementations for enhanced performance. Skills : - Solid foundat...

Responsibilities: DBE engineer with 7-9 yrs exp Hands-on MS SQL, ETL , SSIS Hands-on experience on cloud side and migration exp from AWS and GCP Hands -on experience cloud data platform: Snowflake MS SQL Server, SQLServer Performance Tuning, ANSI SQL, SQL Server,Data Modeling, ETL,CI & CD

Job Summary We are seeking a highly skilled Sr. Developer with 7 to 10 years of experience to join our dynamic team. The ideal candidate will have expertise in Python Databricks SQL Databricks Workflows and PySpark. This role operates in a hybrid work model with day shifts offering the opportunity to work on innovative projects that drive our companys success. Responsibilities Develop and maintain scalable data processing systems using Python and PySpark to enhance data analytics capabilities. Collaborate with cross-functional teams to design and implement Databricks Workflows that streamline data operations. Optimize Databricks SQL queries to improve data retrieval performance and ensure ef...

Experience : 6+ years as Azure Data Engineer including at least 1 E2E Implementation in Microsoft Fabric. Responsibilities : - Lead the design and implementation of Microsoft Fabric-centric data platforms and data warehouses. - Develop and optimize ETL/ELT processes within the Microsoft Azure ecosystem, effectively utilizing relevant Fabric solutions. - Ensure data integrity, quality, and governance throughout Microsoft Fabric environment. - Collaborate with stakeholders to translate business needs into actionable data solutions. - Troubleshoot and optimize existing Fabric implementations for enhanced performance. Skills : - Solid foundational knowledge in data warehousing, ETL/ELT processes...

helps employees unleash their potential while creating value for themselves and the company. Additionally, everyone has individual work and personal needs. We seek to enable the best work environment that helps you and the business work together to produce superior results.

We are seeking a highly skilled Data Engineer with strong expertise in Databricks, ETL processes, Azure Cloud, and Python to join our dynamic team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and architectures to support our data-driven initiatives. Key Responsibilities: Design and Develop Data Pipelines: Create and optimize scalable data pipelines on Databricks to process and analyze large datasets. ETL Processes: Implement and manage ETL processes to ensure data is accurately extracted, transformed, and loaded from various sources ,DLT Framework: Utilize the Delta Live Tables (DLT) framework to build reliable, maintainable, and...

About the Role : We are seeking a highly experienced and proactive SAP Cloud Data Architect to join our team for an urgent 12-month contract. This is a critical role for an individual who thrives in a fast-paced environment and can hit the ground running. You will be instrumental in leading complex data integration initiatives and optimizing our SAP data landscape. Key Responsibilities : - SAP ECC/BW/HANA Solution/Data Architecture : Design and implement end-to-end SAP ECC, BW, and HANA data architectures, ensuring the delivery of scalable, robust, and high-performance solutions. - Data Model & ETL Optimization : Develop, optimize, and maintain data models, ETL processes, and reporting frame...

Role : Minimum 10 years of experience and 8 years in Markit EDM mandatory. We are seeking a highly skilled and motivated Senior MarkIT EDM Data Engineer to join our team. The ideal candidate will be a self-starter who can work independently and collaborate effectively with both internal and external stakeholders. This role requires a deep understanding of data engineering principles, strong technical skills, and the ability to drive projects to successful completion. Key Responsibilities. - Minimum 10 years of experience and 8 years in Markit EDM mandatory. - Design, develop, and maintain data solutions using MarkIT EDM. - Collaborate with internal teams and external stakeholders to gather r...

Role Responsibilities : - Design and implement data pipelines using MS Fabric. - Develop data models to support business intelligence and analytics. - Manage and optimize ETL processes for data extraction, transformation, and loading. - Collaborate with cross-functional teams to gather and define data requirements. - Ensure data quality and integrity in all data processes. - Implement best practices for data management, storage, and processing. - Conduct performance tuning for data storage and retrieval for enhanced efficiency. - Generate and maintain documentation for data architecture and data flow. - Participate in troubleshooting data-related issues and implement solutions. - Monitor and...

We are seeking a highly skilled and motivated Senior MarkIT EDM Data Engineer to join our team. The ideal candidate will be a self-starter who can work independently and collaborate effectively with both internal and external stakeholders. This role requires a deep understanding of data engineering principles, strong technical skills, and the ability to drive projects to successful completion. Key Responsibilities : - Design, develop, and maintain data solutions using MarkIT EDM. - Collaborate with internal teams and external stakeholders to gather requirements and deliver data solutions. - Ensure data quality, integrity, and security across all data processes. - Optimize data workflows and ...

Senior Data Engineer: Design, build, & optimize data platforms on AWS. Expertise in Python, Scala, SQL, Redshift, Terraform, Docker, & Jenkins. Skills include Glue, Lambda, Apache Airflow, Spark, & Databricks. Join our Hyderabad team.

We are seeking a highly skilled and motivated Senior MarkIT EDM Data Engineer to join our team. The ideal candidate will be a self-starter who can work independently and collaborate effectively with both internal and external stakeholders. This role requires a deep understanding of data engineering principles, strong technical skills, and the ability to drive projects to successful completion. Key Responsibilities : - Design, develop, and maintain data solutions using MarkIT EDM. - Collaborate with internal teams and external stakeholders to gather requirements and deliver data solutions. - Ensure data quality, integrity, and security across all data processes. - Optimize data workflows and ...

We are seeking a detail-oriented Data Analyst who will ensure that organizational data is accurately transformed, validated, and prepared for downstream consumption.