210 Spark Streaming Jobs - Page 5

At PwC, individuals in managed services focus on providing outsourced solutions and supporting clients across various functions. By managing key processes and functions on behalf of organisations, they help streamline operations, reduce costs, and enhance efficiency. Skilled in project management, technology, and process optimization, they deliver high-quality services to clients. Those specializing in managed service management and strategy at PwC concentrate on transitioning and running services, managing delivery teams, programmes, commercials, performance, and delivery risk. The role involves continuous improvement, optimizing managed services processes, tools, and services. Your focus l...

Long Description Experienceand Expertise inany of the followingLanguagesat least 1 of them : Java, Scala, Python Experienceand expertise in SPARKArchitecture Experience in the range of 6-10 yrs plus Good Problem SolvingandAnalytical Skills Ability to Comprehend the Business requirementand translate to the Technical requirements Good communicationand collaborative skills with fellow teamandacross Vendors Familiar with development of life cycle includingCI/CD pipelines. Proven experienceand interested in supportingexistingstrategicapplications Familiarity workingwithagile methodology Mandatory Skills: Scala programming.: Experience: 5-8 Years.

We are seeking an experienced Software Engineer with deep expertise in Scala programming and Big Data technologies to design, develop, and maintain large-scale distributed data processing systems. The ideal candidate will be a hands-on developer with a strong understanding of data pipelines, Spark ecosystem, and related technologies, capable of delivering clean, efficient, and scalable code in an Agile environment. Key Responsibilities Develop and maintain scalable, efficient, and robust data processing pipelines using Scala and Apache Spark (Spark Core, Spark SQL, Spark Streaming). Write clean, maintainable, and well-documented Scala code following industry best practices and coding standar...

We Are Hiring Databricks Data Architect | Navi Mumbai (Onsite) Are you passionate about designing scalable, enterprise-grade data platforms? Join Celebal Technologies and work on cutting-edge Azure Databricks solutions in the manufacturing and energy sectors! Role: Databricks Data Architect Experience: 10+ Years Location: Navi Mumbai (Onsite) Notice Period: Immediate to 30 Days Preferred About the Role We are looking for an experienced Databricks Data Architect with strong expertise in Azure Databricks, data modeling, and big data solutions. Youll be responsible for architecting scalable, cloud-native data platforms, enabling real-time and batch processing for advanced analytics and AI-drive...

We are seeking a highly skilled Data Engineer with strong hands-on experience in Spark Streaming, PySpark, Python, and Scala. The ideal candidate will have solid working knowledge of Databricks and cloud platforms, preferably AWS (Azure is acceptable). The role involves building scalable data pipelines, processing real-time data, and supporting advanced analytics solutions. Strong problem-solving skills and the ability to work in a fast-paced environment are essential. Key Skills: Spark Streaming PySpark, Python, Scala Databricks AWS (preferred) or Azure ETL/Data Pipeline Development Big Data Processing Experience Required: 3+ years in data engineering or related field.

About the Role: We are looking for a highly motivated Senior Software Engineer, Data Analytics with experience to join our fast-paced engineering team. The ideal candidate takes full ownership of their work, thrives in cross-functional collaboration, and is passionate about building scalable, fault-tolerant big data systems. In this role, you will design and develop high-performance data platforms, mentor junior engineers, and contribute to delivering impactful analytics solutions that drive strategic business decisions. What Youll Do: Design, build, and optimize scalable and fault-tolerant Big Data pipelines for batch and streaming workloads. Develop real-time streaming applications using A...

About the Role Zywave is seeking a Technical Lead Data Engineering (TurboRater RQR: CPQ Rating) with expertise in Snowflake, SQL Server, and modern data architecture principles including Medallion Architecture and Data Mesh . This role will play a critical part in the TurboRater RQR: CPQ Rating initiative , leading the design and implementation of scalable, secure, and high-performance data pipelines that power rating and CPQ (Configure, Price, Quote) capabilities. The ideal candidate will combine deep technical expertise with insurance domain knowledge to drive innovation and deliver business impact. Key Responsibilities Lead end-to-end design and development of data pipelines supporting Tu...

The Team The Data Engineering team is responsible for architecting, building, and maintaining our evolving data infrastructure, as well as curating and governing the data assets created on our platform. We work closely with various stakeholders to acquire, process, and refine vast datasets, focusing on creating scalable and optimized data pipelines. Our team possesses broad expertise in critical data domains, technology stacks, and architectural patterns. We foster knowledge sharing and collaboration, resulting in a unified strategy and seamless data management. The Impact: This role is the foundation of the products delivered. The data onboarded is the base for the company as it feeds into ...

Job Summary: Supports, develops and maintains a data and analytics platform. Effectively and efficiently process, store and make data available to analysts and other consumers. Works with the Business and IT teams to understand the requirements to best leverage the technologies to enable agile data delivery at scale. Key Responsibilities: Implements and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured). Implements methods to continuously monitor and troubleshoot data quality and data integrity issues. Implements data governance processes and methods for managing metadata, access, retention ...

Position Overview We are seeking a skilled and experienced Senior PySpark Developer with expertise in Apache spark, Spark Batch, and Spark Streaming to join our dynamic team. The ideal candidate will design, develop, and maintain high-performance, scalable applications for processing large-scale data in batch and real-time environments. Required Skills and Qualifications Experience : 7+ years of professional experience in PySpark development. Technical Skills : Strong proficiency in PySpark Deep understanding of Apache Spark architecture, distributed computing concepts and parallel processing. Proven experience in building and optimizing complex ETL pipeline using PySpark Experience with var...

You will be working as an offshore Senior Developer in Chennai with expertise in Databricks and a willingness to adapt to new technologies. You will collaborate effectively with the team and the position involves replacing an existing Senior Developer. This role is long-term and may be renewed annually. As a team player, you must possess a minimum of 5 years of experience in IT development along with strong analytical and problem-solving skills. Your responsibilities will include designing solutions, conducting code reviews, and providing guidance to junior engineers. Proficiency in SQL, backend development, and experience in data-driven projects is essential. Your expertise should cover Pyt...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture, and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. As a Staff - Data Engineer at EY, your responsibilities include designing and developing software components using various tools such as PySpark, Sqoop, Flume, Azure Databricks, and more. You will perform detailed analysis and effectively interact with onshore/offshore team members, ensuring all deliverables conform to the highest quality st...

As a Senior AWS Data Engineer Cloud Data Platform at Teamware Solutions, a division of Quantum Leap Consulting Pvt. Ltd, located in Bangalore, you will be responsible for end-to-end implementation of Cloud data engineering solutions like Enterprise Data lake and Data hub in AWS. Working onsite in an office environment for 5 days a week, you will collaborate with the Offshore Manager and Onsite Business Analyst to understand the requirements and deliver scalable, distributed, cloud-based enterprise data solutions. You should have a strong background in AWS cloud technology, with 4-8 years of hands-on experience. Proficiency in architecting and delivering highly scalable solutions is a must, a...

About the job: At Citi, we're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with a laser focus on evolving the ways of doing things. As one of the world's most global banks, we're changing how the world does business. Shape your Career with Citi We're currently looking for a high-caliber professional to join our team as AVP- Data Engineer based in Pune, India. Being part of our team means that we'll provide you with the resources to meet your unique needs, empower you to make healthy ...

At EY, you will have the opportunity to shape a career as unique as you are, supported by a global network, inclusive culture, and cutting-edge technology to help you reach your full potential. Your individual perspective and voice are valued to contribute to the continuous improvement of EY. By joining us, you can create an outstanding experience for yourself while contributing to a more efficient and inclusive working world for all. As a Data Engineering Lead, you will work closely with the Data Architect to design and implement scalable data lake architecture and data pipelines. Your responsibilities will include designing and implementing scalable data lake architectures using Azure Data...

As a Solution Designer (Cloud Data Integration) at Barclays within the Customer Digital and Data Business Area, you will play a vital role in supporting the successful delivery of location strategy projects. Your responsibilities will include ensuring projects are delivered according to plan, budget, quality standards, and governance protocols. By spearheading the evolution of the digital landscape, you will drive innovation and excellence, utilizing cutting-edge technology to enhance our digital offerings and deliver unparalleled customer experiences. To excel in this role, you should possess hands-on experience working with large-scale data platforms and developing cloud solutions within t...

The role of Lead, Software Engineer at Mastercard involves playing a crucial part in the Data Unification process across different data assets to create a unified view of data from multiple sources. This position will focus on driving insights from available data sets and supporting the development of new data-driven cyber products, services, and actionable insights. The Lead, Software Engineer will collaborate with various teams such as Product Manager, Data Science, Platform Strategy, and Technology to understand data needs and requirements for delivering data solutions that bring business value. Key responsibilities of the Lead, Software Engineer include performing data ingestion, aggrega...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. We are counting on your unique voice and perspective to help EY become even better. Join us and build an exceptional experience for yourself, and a better working world for all. We are seeking a highly skilled and motivated Data Analyst with experience in ETL services to join our dynamic team. As a Data Analyst, you will be responsible for data requirement gathering, preparing data requirement artefacts, data integration strategies, data quality, data cleansing, optimizing data pipelines, and solutions that support business ...

BizViz is a company that offers a comprehensive view of a business's data, catering to various industries and meeting the diverse needs of business executives. With a dedicated team of over 50 professionals working on the BizViz platform for several years, the company aims to develop technological solutions that provide our clients with a competitive advantage. At BizViz, we are committed to the success of our customers, striving to create applications that align with their unique visions and requirements. We steer clear of generic ERP templates, offering businesses a more tailored solution. As a Big Data Engineer at BizViz, you will join a small, agile team of data engineers focused on buil...

Greetings from Infogain! We are having Immediate requirement for Big Data Engineer (Lead) position in Infogain India Pvt ltd. As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers. You will work closely with clients and team members to understand their requirements and develop architectures that meet their needs. You will also be responsible for providing technical leadership and guidance to your team. Mode of Hiring-Permanent Skills : (Azure OR AWS) AND Apache Spark OR Hive OR Hadoop AND Spark Streaming OR Apache Flink OR Kafka AND NoSQL AND Shell OR Python. Exp: 10 to 14 years Location: Bangalore/Noida/Gurgaon/Pune/Mumbai/Kochi Notice period- Early...

About the Team As a part of the DoorDash organization, you will be joining a data-driven team that values timely, accurate, and reliable data to make informed business and product decisions. Data serves as the foundation of DoorDash's success, and the Data Engineering team is responsible for building database solutions tailored to various use cases such as reporting, product analytics, marketing optimization, and financial reporting. By implementing robust data structures and data warehouse architecture, this team plays a crucial role in facilitating decision-making processes at DoorDash. Additionally, the team focuses on enhancing the developer experience by developing tools that support th...

The Applications Development Intermediate Programmer Analyst position is an intermediate level role where you will be responsible for participating in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your main objective will be to contribute to applications systems analysis and programming activities. You will be responsible for developing and maintaining Java-based applications using the Spring framework. Additionally, you will design and implement batch processing solutions using Spark Batch for large-scale data processing and build real-time data pipelines using Spark Streaming for processing streaming data. ...

As an AWS Big Data Engineer working in a remote location, you will be a crucial part of a company that provides enterprise-grade platforms designed to accelerate the adoption of Kubernetes and Data. Our flagship platform, Gravity, offers developers a simplified Kubernetes experience by handling all the underlying complexities. You will have the opportunity to utilize tailor-made workflows to deploy Microservices, Workers, Data, and MLOps workloads across multiple Cloud Providers. Gravity takes care of various Kubernetes-related orchestration tasks including cluster provisioning, workload deployments, configuration management, secret management, scaling, and provisioning of cloud services. Ad...

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. Primary Responsibilities: Analyze business requirements & functional specifications Be able to determine the impact...

Role & responsibilities We are seeking a skilled Data Engineer to maintain robust data infrastructure and pipelines that support our operational analytics and business intelligence needs. Candidates will bridge the gap between data engineering and operations, ensuring reliable, scalable, and efficient data systems that enable data-driven decision making across the organization. Strong proficiency in Spark SQL, hands-on experience with realtime Kafka, Flink Databases: Strong knowledge of relational databases (Oracle, MySQL) and NoSQL systems Proficiency with Version Control Git, CI/CD practices and collaborative development workflow Strong operations management and stakeholder communication s...