6525 Data Engineering Jobs - Page 26

Job Title: Software Engineer / Strong Java background Urgent Requirement Experience Required: 10-15 years Job Description We are looking for a skilled Software Engineer with 10-15 years of experience in backend development and distributed systems. The ideal candidate should have strong programming expertise in Python and Java , hands-on experience with messaging systems (RabbitMQ, Kafka) , and working knowledge of infrastructure automation (Chef) . This role involves designing, developing, and maintaining scalable, high-performance applications and ensuring smooth integration between microservices and messaging pipelines. Key Responsibilities: Design, develop, and maintain backend applicatio...

Design, develop, and maintain scalable and efficient big data processing pipelines distributed computing systems. Collaborate with cross-functional teams to understand data requirements and design appropriate data solutions. Implement data ingestion, processing, and transformation processes to support various analytical and machine learning use cases. Optimize and tune data pipelines for performance, scalability, and reliability. Monitor and troubleshoot pipeline performance issues, identifying and resolving bottlenecks. Ensure data quality and integrity throughout the pipeline, implementing data validation and error handling mechanisms. Stay updated on emerging technologies and best practic...

DataBricks Engineer, you will work with multiple teams to deliver solutions on the Azure Cloud using core cloud data warehouse tools. Must be able to analyze data and develop strategies for populating data lakes if required

Company Description Cloud Destinations, headquartered in Silicon Valley, specializes in digital transformation and enterprise application development for the Healthcare, Retail, and Technology sectors. We excel in cloud migrations, multi-cloud operations, DevOps, security operations, and IoT. Our globally distributed team includes skilled technical experts and effective leaders who thrive on teamwork and respect for diverse viewpoints. For more details, visit our website: www.clouddestinations.com. Role Description This is a full-time, on-site role for a Data Engineer Manager located in Coimbatore . We are looking for a highly skilled Data Engineering Manager to lead and grow our data engine...

About Client Hiring for One of the Most Prestigious Multinational Corporations Job Title : Azure Databricks Qualification : Any Graduate or Above Experience : 4 to 8 Yrs Location : Chennai , Bangalore Key Responsibilities: Design and build scalable and robust data pipelines using Azure Databricks, PySpark, and Spark SQL. Integrate data from various structured and unstructured data sources using Azure Data Factory, ADLS, Azure Synapse,etc. Develop and maintain ETL/ELT processes for ingestion, transformation, and storage of data. Collaborate with data scientists, analysts, and other engineers to deliver data products and solutions. Monitor, troubleshoot, and optimize existing pipelines for per...

Roles & Responsibilities We are looking for a candidate with 7+ years of experience in a Data Engineer role, having experience using the following software/tools: Technical skills required for ETL engineer in our project : Experience with big data tools: Spark, Kafka Experience with stream-processing systems: Spark-Streaming, Experience with Azure SQL, relational SQL and NoSQL databases, Experience with data pipeline and workflow management tools: Synapse Pipeline, Azure Data Factory Experience with Azure cloud services: Databricks, Synapse Analytics, Azure Function, ADLS Experience with object-oriented/object function scripting languages: Scala ,Python Experience in Terraform ETL Data Engin...

Role Overview: This role will be responsible for the automation of data pipelines from multiple sources for the development and maintenance of dashboards for tracking Microsoft cloud metrics. This role merges data engineering and business intelligence to provide timely insights into partnership performance. Key responsibilities Automate data extraction from Partner Center into RSM's repository, ensuring regular refreshes and minimal manual steps. Design and maintain an internal Power BI dashboard visualizing key metrics with interactive features. Integrate data from multiple sources for comprehensive reporting. Analyze trends, generate actionable insights, and support ad-hoc queries. Ensure ...

Data Engineer Position Overview Role Summary We are searching for a talented and motivated Data Engineer to join our team The ideal candidate will have expertise in data modeling, analytical thinking, and developing ETL processes using Python In this role, you will be pivotal in transforming raw data from landing tables into reliable, curated master tables, ensuring accuracy, accessibility, and integrity within our Snowflake data platform, Experience Overall 6+ yrs of experience with relevant experience of 4+ years Main Responsibilities Design, Develop, and Maintain ETL Processes: Build and maintain scalable ETL pipelines in Python to extract, transform, and load data into Snowflake master t...

About The Opportunity : Join a pioneering consulting firm in the Data Analytics and Cloud Solutions sector, where transformative data architectures empower global enterprises. We specialize in leveraging cutting-edge Snowflake technologies and innovative cloud solutions to drive real-time insights and business intelligence. This remote role, based in India, offers the opportunity to work on high-impact projects while collaborating with a diverse team of experts. Role & Responsibilities : - Design, implement, and optimize scalable data warehousing solutions using Snowflake Cortex. - Develop robust ETL pipelines that ensure data quality, reliability, and efficient integration across platforms....

Department : Platform Engineering Summary : We are seeking a highly skilled Data Engineer with expertise in ontology development and knowledge graph implementation. This role will be pivotal in shaping our data infrastructure and ensuring the accurate representation and integration of complex data sets. You will leverage industry best practices, including the Basic Formal Ontology (BFO) and Common Core Ontologies (CCO), to design, develop, and maintain ontologies, semantic and syntactic data models, and knowledge graphs on the Databricks Data Intelligence Platform that drive data-driven decision-making and innovation within the company. Responsibilities : Ontology Development : - Design and ...

Experience : 6+ years as Azure Data Engineer including at least 1 E2E Implementation in Microsoft Fabric. Responsibilities : - Lead the design and implementation of Microsoft Fabric-centric data platforms and data warehouses. - Develop and optimize ETL/ELT processes within the Microsoft Azure ecosystem, effectively utilizing relevant Fabric solutions. - Ensure data integrity, quality, and governance throughout Microsoft Fabric environment. - Collaborate with stakeholders to translate business needs into actionable data solutions. - Troubleshoot and optimize existing Fabric implementations for enhanced performance. Skills : - Solid foundational knowledge in data warehousing, ETL/ELT processes...

We at Blackstraw.ai. are organizing a Walk-in Interview Drive for Data Scientist with minimum 4 years exp in Data Engineer Mini 4 Years Exp in PySpark, Spark, Machine Learning, Azure Data Bricks Python Mini 4 Years Strong experience in Python, Flask, RestAPI, Fast API, Job Requirements Minimum of 5+ years working experience with Spark, Hadoop eco systems. Minimum of 4+ years working experience on designing data streaming pipelines. Should have experience in Data Ingestion and Integration into data lake using hadoop ecosystem tools such as Sqoop, Spark, SQL, Hive, Airflow, etc.. Should have experience optimizing (Performance tuning) data pipelines. Create and maintain a scalable infrastructur...

Inviting applications for the role of Principal Consultant, Process Mining: Pre- Sales, Data Mining, Analytics & Delivery! Looking for immediate to 30 days' joiners only Responsibilities: • Client Engagement • Educate clients (and internal Genpact stakeholders) in the benefits of process mining technology combined with domain and change management expertise • Inspire clients with digital transformation strategies that can be deployed to help achieve their goals across their entire organizations • Present demonstrations of the technology in action • Work closely with client stakeholders (senior execs, business process owners, technology), Genpact stakeholders (sales, consultants, operating le...

Skills: 1] Hands-on experience in LLM/ML based application development 2] Productionizing LLM/ML apps Data Handling/processing/engineering Dataset creation/curation 3] Understanding of different LLM performance metrics, fine-tuning, prompt engineering 4] Image/Video Processing 5] Generative AI, OpenAI, Claude Knowledge. Good in Prompt Engineering, autogen or similar agentic framework knowledge Skills : - LLM/ML ,application development, Data Handling,Data processing,Data engineering,Dataset creation, LLM performance metrics, fine-tuning, prompt engineering, Image/Video Processing, Generative AI, OpenAI, Claude, Prompt Engineering, autogen, agentic framework Keywords :application development,...

SUMMARY Key Responsibilities: Work closely with clients to understand their business requirements and design data solutions that meet their needs. Develop and implement end-to-end data solutions that include data ingestion, data storage, data processing, and data visualization components. Design and implement data architectures that are scalable, secure, and compliant with industry standards. Work with data engineers, data analysts, and other stakeholders to ensure the successful delivery of data solutions. Participate in presales activities, including solution design, proposal creation, and client presentations. Act as a technical liaison between the client and our internal teams, providing...

Job Application Link: https://app.fabrichq.ai/jobs/b41f52d2-09e3-4f5e-9bf2-c587f6a0551f Job Summary: Junior Data Engineer role at Aays Analytics focused on designing and implementing data and analytics solutions on Google Cloud Platform (GCP). The position involves working with clients to reinvent their corporate finance functions through advanced analytics. Key responsibilities include architecture design, data modeling, and mentoring teams on GCP-based solutions. Key Responsibilities Design and drive end-to-end data and analytics solution architecture from concept to delivery on Google Cloud Platform (GCP) Design, develop, and support conceptual, logical, and physical data models for advan...

SUMMARY About the Role: We are looking for a Senior Data Engineer to lead the design and implementation of scalable data infrastructure and engineering practices. This role will be critical in laying down the architectural foundations for advanced analytics and AI/ML use cases across global business units. You’ll work closely with the Data Science Lead, Product Manager, and other cross-functional stakeholders to ensure data systems are robust, secure, and future-ready. Key Responsibilities: Architect and implement end-to-end data infrastructure including ingestion, transformation, storage, and access layers to support enterprise-scale analytics and machine learning. Define and enforce data e...

Role Purpose The purpose of the role is to create exceptional architectural solution design and thought leadership and enable delivery teams to provide exceptional client engagement and satisfaction. Do 1.Develop architectural solutions for the new deals/ major change requests in existing deals Creates an enterprise-wide architecture that ensures systems are scalable, reliable, and manageable. Provide solutioning of RFPs received from clients and ensure overall design assurance Develop a direction to manage the portfolio of to-be-solutions including systems, shared infrastructure services, applications in order to better match business outcome objectives Analyse technology environment, enter...

Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client queries Resolve cl...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

NTT DATA Services currently seeks Python Hadoop Developer to join our team in Responsibilities Skillset: Python, Hadoop, ETL, RDBMS, Unix, GCP Vertex AI Responsibilities: Data Ingestion Pipelines, AIML Model deployments, Data Engineering, ML Engineering with Python.

Key Skills required The role involves applying advanced analytics, machine learning, and statistical modeling to optimize plant operations, enhance process efficiency, improve product quality, ensure sustainability, and reduce costs. Knowledge in OSI-PI Software and Data Analytics Platform, such as Seeq, Aspen Tech, and Python, is preferred. Role & responsibilities Collect, clean, and analyze large-scale structured and unstructured data from plant operations (sensors, DCS/SCADA systems, ERP, MES, laboratory data). Develop predictive and prescriptive models for: Process optimization (yield, throughput, energy efficiency). Equipment reliability and predictive maintenance. Quality control and a...

Were Hiring Engineering Lead (Cloud & Data) | Data Engineer SME Location: Hinjewadi, Pune Work Mode: Hybrid (2 - 3 days from office) Experience: 12 - 17 Years Key Skills: Cloud Platforms (AWS preferred) Data Engineering (Databricks, Spark, Python, SQL) Application & Data Integration in Cloud ETL, Data Pipelines, Airflow Strong Leadership & Mentorship skills

As a Snowflake Lead with experience and additional expertise in ETL tools and DBT, you will play a crucial role in designing, implementing, and managing Snowflake solutions to facilitate efficient data management and analytics. Your leadership will be pivotal in guiding the team to deliver high-quality, scalable, and optimized data solutions. Your proficiency in ETL and DBT will further enhance the organization's data processing capabilities. **Roles & Responsibilities:** - Perform analysis on the existing Data Storage system and develop Data Solutions in Snowflake. - Design the data warehouse and provide guidance to the team for implementation using Snowflake SnowSQL. - Define and communica...

Role Overview: You will be responsible for designing, building, and deploying Generative AI models using foundational models like GPT, BERT, LLaMA, PaLM, etc. Additionally, you will develop scalable GenAI applications, integrate them with enterprise systems using APIs and SDKs, and fine-tune large language models (LLMs) for domain-specific use cases. You will also design, implement, and manage end-to-end machine learning pipelines on Microsoft Azure, collaborate with data scientists to productionize ML models, and automate deployment and monitoring of models using CI/CD pipelines and Azure DevOps tools. Furthermore, you will ensure security, compliance, and governance of ML workflows and dat...