47 Streaming Data Jobs

Health for a Better World Lead Software Engineer, Providence India About Providence At Providence, we are grounded in our goal toserve allas we engineer the future of healthcare . Providence Global Innovation Center, launched in Feb 2020, is the first development and innovation center of Providence outside the United States. Providence is a $26B non-profit organization and is one of the largest health systems in the United States. To read more, Why Us Best In-class Benefits Inclusive Leadership Challenging Work Competitive Pay Employee Friendly Policies Location: Hyderabad Organization background We're committed to empowering clinical caregivers by simplifying healthcare and making it more a...

The IT Technical Engineer Act as a technical software expert, accountable for software development including the research, design, programming, and testing of operating or new software. Supporting business and IT to adapt digital technologies faster and to improve application or solution performance. HOW YOU WILL CONTRIBUTE AND WHAT YOU WILL LEARN Develop ETL/ELT workflows for both batch and streaming data, supporting structured and unstructured data sources. Implement and support Lakehouse and Medallion architecture (Bronze/Silver/Gold) using Azure Data Lake Storage and Delta Lake. Optimize Spark jobs through effective partitioning, caching, cluster tuning, and cost-efficient resource utili...

Spark/Scala - Job Description Qualifications: Bachelor of Engineering Desired Skills: Scala, Spark, Unix Must-Have Skills Minimum 4+ years of experience in development of Spark Scala. Experience in designing and developing solutions for Big Data using Hadoop ecosystem technologies such as HDFS, Spark, Hive, Parquet File format, YARN, MapReduce, Sqoop. Good experience in writing and optimizing Spark jobs, Spark SQL, and handling both batch and streaming data processing. Experience in writing and optimizing complex Hive and SQL queries to process large datasets, including proficiency with UDFs, tables, joins, and views. Experience in debugging Spark code. Working knowledge of basic UNIX comman...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, collaborating with Integration Architects and Data Architects to ensure cohesive integration between sys...

Role Overview: You are a seasoned Data/Information Management Senior Analyst with in-depth disciplinary knowledge and expertise. You will contribute to the development of new techniques, improve processes, and workflows within the area or function. Your role involves integrating subject matter expertise and industry knowledge within a defined area. You must have a deep understanding of how different areas integrate within the sub-function, coordinate effectively, and contribute to the function's objectives and overall business goals. Your analytical skills will be crucial in filtering, prioritizing, and validating complex material from multiple sources. Excellent communication, diplomacy, an...

Join us as a Data Engineer Were looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secure Day-to-day, youll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through insights If youre ready for a new challenge, and want to bring a competitive edge to your career profile by delivering streaming data ingestions, this could be the role for you We're offering this role at associate vice president level What youll do Your daily responsibilities will include you developing a comprehensive knowledge of our data structures and me...

Overview We are looking for a Lead Software Engineer to join an award-winning team with a proven track record of combining data science techniques with an intimate knowledge of payments data to aid Financial Institutions in their fight against money laundering and fraud. We craft bespoke services that help our clients gain an understanding of the underlying criminal behaviour that drives financial crime, empowering them to take action. As part of the application development team, your role will focus on creating and maintaining products across the whole lifecycle. Role Establish and enforce best practices for microservices architecture, ensuring scalability, reliability, and maintainability ...

Senior Data Scientist with 3-5 years of hands-on industry experience in machine learning (ML) and natural language processing (NLP). In this role, you will help design, develop, and maintain document intelligence platforms that power multiple product streams across the organization. As a Senior Data Scientist, you will collaborate closely with data engineers, domain experts, and fellow data scientists to unlock large-scale data capabilities. You will work with both structured and unstructured clinical datasets, applying advanced algorithms and state-of-the-art modeling techniques to build, optimize, and deploy scalable ML/NLP solutions in production environments. Your contributions will dire...

Join us as a Data Engineer Were looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secure Day-to-day, youll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through insights If youre ready for a new challenge, and want to bring a competitive edge to your career profile by delivering streaming data ingestions, this could be the role for you We're offering this role at associate vice president level What youll do Your daily responsibilities will include you developing a comprehensive knowledge of our data structures and me...

What youll do Your daily responsibilities will include you developing a comprehensive knowledge of our data structures and metrics, advocating for change when needed for product development. Youll also provide transformation solutions and carry out complex data extractions. Well expect you to develop a clear understanding of data platform cost levels to build cost-effective and strategic solutions. Youll also source new data by using the most appropriate tooling before integrating it into the overall solution to deliver it to our customers. Youll also be responsible for: Driving customer value by understanding complex business problems and requirements to correctly apply the most appropriate...

Project description Luxoft has been asked to contract a Developer in support of a number of customer initiatives. The primary objective is to develop based on client requirements in the Telecom/network work environment Responsibilities A Data Engineer with experience in the following techologies: Databricks and Azure Apache Spark-based, hands on Python, SQL, Apache Airflow. Databricks clusters for ETL processes. Integration with ADLS, Blob Storage. Efficiently ingest data from various sources, including on-premises databases, cloud storage, APIs, and streaming data. Use Azure Key Vault for managing secrets. Hands on experience working with API's Kafka/Azure EventHub streaming hands on experi...

As a Business Process Analyst at our company, you will play a crucial role in designing and maintaining analytics frameworks for business processes and financial workflows. Your responsibilities will include developing dashboards and reports to offer real-time insights into key business metrics. You will also analyze user interaction patterns with AI features to enhance adoption and effectiveness. Managing ETL pipelines to ensure data quality and accessibility, as well as generating actionable insights from complex financial datasets to inform product strategy will be part of your daily tasks. Key Responsibilities: - Designing and maintaining analytics frameworks for business processes and f...

Join us as a Data Engineer Were looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secure Day-to-day, youll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through insights If youre ready for a new challenge, and want to bring a competitive edge to your career profile by delivering streaming data ingestions, this could be the role for you We're offering this role at associate vice president level What youll do Your daily responsibilities will include you developing a comprehensive knowledge of our data structures and me...

Project description We need an experienced Python Developer to work for a leading investment bank client. Responsibilities Design, develop, and maintain full-stack Python applications with modern frontend frameworks Build and optimize RAG (Retrieval-Augmented Generation) systems for AI applications Create and implement efficient vector databases and knowledge stores Develop APIs that connect frontend interfaces with backend AI services Implement and maintain CI/CD pipelines for AI applications Monitor application performance and troubleshoot issues in production Skills Must have Bachelor's degree in Computer Science, Software Engineering, or related field 9+ years of experience in full-stack...

Job Summary: We are seeking a highly skilled Senior Java Developer with expertise in building high-performance, scalable systems. The ideal candidate will have hands-on experience with the latest Java versions (Java 17 or Java 21) , Spring Boot 3.3 , and advanced system architectures like streaming data and message bus-based systems . Experience with Kafka and exposure to trading systems is highly desirable. Key Responsibilities: Develop, test, and deploy high-performance applications using Java (Java 17/Java 21) and Spring Boot 3.3 . Design and implement RESTful services with robust and scalable APIs. Work with streaming data architectures and message bus systems for real-time data processi...

Data Engineer to build and manage data pipelines that support batch and streaming data solutions. The role requires expertise in creating seamless data flows across platforms like Data Lake/Lakehouse in Cloudera, Azure Databricks, Kafka for both batch and stream data pipelines etc. Responsibilities Strong experience in develop, test, and maintain data pipelines (batch & stream) using Cloudera, Spark, Kafka and Azure services like ADF, Cosmos DB, Databricks, NoSQL DB/ Mongo DB etc. Strong programming skills in spark, python or scala & SQL. Optimize data pipelines to improve speed, performance, and reliability, ensuring that data is available for data consumers as required. Create ETL pipeline...

Project description We need a Python Developer to work for a leading investment bank client. Responsibilities Design, develop, and maintain full-stack Python applications with modern frontend frameworks Build and optimize RAG (Retrieval-Augmented Generation) systems for AI applications Create and implement efficient vector databases and knowledge stores Develop APIs that connect frontend interfaces with backend AI services Implement and maintain CI/CD pipelines for AI applications Monitor application performance and troubleshoot issues in production Skills Must have Bachelor's degree in Computer Science, Software Engineering, or related field 4+ years of experience in full-stack Python devel...

2+ years of implementation experience with Adobe Experience Cloud products especially Adobe Experience Platform and Journey Optimizer Expertise in deploying, configuring and optimizing all major Experience Platform services and Journey Optimizer features Strong SQL skills for querying datasets, implementing data transformations, cleansing data, etc Hands-on experience developing custom applications, workflows and integrations using Experience Platform APIs Deep familiarity with Adobe Experience Platform and Journey Optimizer technical implementations including: Setting up source connectors and ingesting data using the API and UI Configuring Experience Events (XDM schemas) for capturing data ...

The Role : Mastercard Commercial Solutions is looking for an Engineering Director and a Technology leader who will lead engineering teams focused on our core services someone who combines deep technical experience with great people management skills and can help elevate our teams and our approach to engineering. We are looking for a candidate that has proven experience in coding, architecture leadership, and leading engineers in a fast-paced environment. We are seeking a results-driven Director to lead and grow our data engineering team. This leader will establish the data vision, strategy, and roadmap for the Commercial Solutions product, aligned to broader enterprise data strategy and plat...

Join us as a Data Engineer Youll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-making Well look to you to drive the build of effortless, digital first customer experiences If youre ready for a new challenge and want to make a far-reaching impact through your work, this could be the opportunity youre looking for We're offering this role at vice president level What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and se...

Join us as a Data Engineer Youll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-making Well look to you to drive the build of effortless, digital first customer experiences If youre ready for a new challenge and want to make a far-reaching impact through your work, this could be the opportunity youre looking for We're offering this role at vice president level What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and se...

Title: Technical Lead (AI/ML/GenAI) Job Description We are seeking a highly skilled and innovative Technical Lead ( AI/ML/GenAI) with strong hands-on experience in Agentic AI, Machine Learning, and Cloud Platforms like AWS and Databricks. The ideal candidate will be proficient in building intelligent systems using agentic frameworks to deliver scalable, production-grade solutions such as chatbots and autonomous agents. Key Responsibilities Lead the design, development, and deployment of advanced machine learning models and algorithms for various applications. Build and optimize chatbots and autonomous agents using LLM endpoints and frameworks like LangChain, Semantic Kernel, or similar. Impl...

What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and secure. Youll drive customer value by understanding complex business problems and requirements to correctly apply the most appropriate and reusable tool to gather and build data solutions. Youll support our strategic direction by engaging with the data engineering community to deliver opportunities, along with carrying out complex data engineering tasks to build a scalable data architecture. Your responsibilities will al...

About The Role Project Role : Application Support Engineer Project Role Description : Act as software detectives, provide a dynamic service identifying and solving issues within multiple components of critical business systems. Must have skills : Data Engineering Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Key Responsibilities Design and build efficient data ingestion pipelines from multiple sources using Databricks and Spark.Develop and implement strategies that integrate data ingestion, processing, governance, and analytics.Store and process raw, unstructured, semi-structured, and structured data in Databrick...

About The Company Tata Communications Redefines Connectivity with Innovation and IntelligenceDriving the next level of intelligence powered by Cloud, Mobility, Internet of Things, Collaboration, Security, Media services and Network services, we at Tata Communications are envisaging a New World of Communications Job Title: Group Product Manager/ Deputy General Manager Location: Bengaluru, India (Hybrid) Employment Type: Full-time Job Job Summary ************************************ Tata Communications is seeking a visionary Group Product Manager to lead the development of our AI-enabled Data Platform Cloud Services within the Vayu Cloud PaaS team. This role is central to building India's firs...