210 Spark Streaming Jobs - Page 3

Primary Skill: Spark, Java programming Strong knowledge in Apache Spark framework Core Spark, Spark Data Frames, Spark streaming Hands-on experience in any one of the programming languages (Java, Scala) Good understanding of distributed programming concepts. Experience in optimizing Spark DAG, and Hive queries on Tez Experience using tools like Git, Autosys, Bitbucket, Jira Mandatory Skills: Apache Spark.Experience: 5-8 Years.

Position Overview We are seeking a skilled and experienced Senior PySpark Developer with expertise in Apache spark, Spark Batch, and Spark Streaming to join our dynamic team. The ideal candidate will design, develop, and maintain high-performance, scalable applications for processing large-scale data in batch and real-time environments. Required Skills and Qualifications Experience : 7+ years of professional experience in PySpark development. Technical Skills : Strong proficiency in PySpark Deep understanding of Apache Spark architecture, distributed computing concepts and parallel processing. Proven experience in building and optimizing complex ETL pipeline using PySpark Experience with var...

Key Responsibilities : Design, implement, and maintain data science pipelines from data ingestion to model deployment. Collaborate with data scientists to operationalize ML models and algorithms in production environments. Develop robust APIs and services for ML model inference and integration. Build and optimize large-scale data processing systems using Spark, Pandas, or similar tools. Ensure data quality and pipeline reliability through rigorous testing, validation, and monitoring. Work with cloud infrastructure (AWS) for scalable ML deployment. Manage model versioning, feature engineering workflows, and experiment tracking. Optimize performance of models and pipelines for latency, cost, a...

Skill Set : Java Developer Experience : 3 to 6 years OS : Good Understanding and working experience on Linux. Cloud : Exposure in AWS or similar cloud environments Exposure in Docker, K8s Hands on experience on linux shell scripting Languages : Java ajax, react js or node js javascript/html Frameworks : spring , hibernate or any jpa Appserver : Apache tomcat Exposure to SPARK Streaming with Scala is an added advantage Scripting : hands on experience in bash or some shell scripting. Database : Exposure to some database like mysql or any rdbms Roles and Responsibilities Key Roles & Responsibilities: Executing automated test iterations, tracking and reporting test results, troubleshooting and c...

As a PySpark Developer at our company, you will be responsible for developing and maintaining data processing pipelines using PySpark, Apache Spark's Python API. You will collaborate with data engineers, data scientists, and other stakeholders to design and implement scalable and efficient data processing solutions. Key Responsibilities: - Designing, developing, and maintaining PySpark data processing pipelines for processing structured and unstructured data - Collaborating with data engineers and data scientists to understand data requirements and design efficient data models and transformations - Optimizing and tuning PySpark jobs for performance, scalability, and reliability - Implementin...

As a highly skilled Azure PySpark Solution Architect, your role involves designing and implementing scalable data solutions on Microsoft Azure. Your expertise in Azure services, PySpark, and solution architecture will ensure efficient data processing and analytics workflows for enterprise applications. **Key Responsibilities:** - Design and implement end-to-end data solutions using Azure Data Services and PySpark - Develop high-performance ETL pipelines leveraging Azure Databricks, Azure Data Factory, and Synapse Analytics - Architect scalable, secure, and cost-efficient cloud solutions aligned with business objectives - Collaborate with data engineers, data scientists, and business stakehol...

Work Location: Pan India Experience: 12+yrs Responsibility: Provide top-quality solution design and execution Provide support in defining the scope and sizing of work Align the organizations big data solutions with their Client initiatives as requested Engage with clients to understand strategic requirements Responsible for translating business requirements into technology solutions Work with domain experts to put together a delivery plan with and stay on track Utilize Big Data technologies to design, develop, and evolve scalable and fault-tolerant distributed components Organize all meetings with customers and ensure prompt resolution of gaps and roadblocks Stay current on latest technology...

Candidate Specification : Any Graduate, Min 5+ years relevant Experience Notice Period = Immediate to 30 days We are seeking a highly skilled Senior Data Engineer with strong expertise in cloud-based data solutions (primarily AWS). The ideal candidate will design and build scalable data pipelines, manage large datasets, and ensure reliable data availability for analytics and business intelligence. You will work closely with data scientists, analysts, and other engineering teams to enable advanced analytics and AI/ML initiatives. Key Responsibilities Design, develop, and optimize scalable ETL/ELT pipelines for structured and unstructured data. Architect and manage data lakes, data warehouses,...

Location: Remote / Pan India,Mumbai, Delhi / NCR, Bengaluru , Kolkata, Chennai, Hyderabad, Ahmedabad, Pune, Notice Period: Immediate iSource Services is hiring for one of their client for the position of Java kafka developer. We are seeking a highly skilled and motivated Confluent Certified Developer for Apache Kafka to join our growing team. The ideal candidate will possess a deep understanding of Kafka architecture, development best practices, and the Confluent platform. You will be responsible for designing, developing, and maintaining scalable and reliable Kafka-based data pipelines and applications. Your expertise will be crucial in ensuring the efficient and robust flow of data across ...

As a Data Analytics focused Senior Software Engineer at PubMatic, you will be responsible for developing advanced AI agents to enhance data analytics capabilities. Your expertise in building and optimizing AI agents using technologies such as Hadoop, Spark, Scala, Kafka, and cloud-based solutions will enable you to improve data-driven insights and analytical workflows. Key Responsibilities: - Build and implement a highly scalable big data platform to process terabytes of data and provide in-depth analytics to customers. - Develop backend services using Java, REST APIs, JDBC, and AWS. - Build and maintain Big Data pipelines using technologies like Spark, Hadoop, Kafka, and Snowflake. - Archit...

Streaming data Technical skills requirements :- Experience- 5+ Years Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred) - Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR - Hands-on experience of programming language like Scala with Spark. - Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases - Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred - Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie - Hands on working experience of data processing at scale with event driven syste...

**Job Description:** As a Java Developer at our company, you will be responsible for developing and maintaining Java applications with a focus on REST API and Spring Boot. Your primary responsibilities will include: - Utilizing your 8 to 10 years of experience in Java, REST API, and Spring Boot to design and implement robust and scalable solutions. - Demonstrating hands-on experience in Angular JS, React JS, or View JS to enhance the user interface and overall user experience. - Collaborating with cross-functional teams to ensure the successful delivery of projects in an agile environment. - Applying your expertise in data cleaning, visualization, and reporting to extract actionable insights...

Location: Pune or Remote (within India) Experience: 5+ years Responsibilities: Design, develop, and maintain high-performance, scalable, and maintainable Scala applications. Develop and maintain RESTful APIs using Scala frameworks Work with relational databases (e.g., MySQL, PostgreSQL) using SQL. Participate in all phases of the software development lifecycle, including requirements gathering, design, development, testing, and deployment. 1 Troubleshoot and debug complex issues in Scala applications. Collaborate effectively with other developers, testers, and product managers. Stay up-to-date with the latest advancements in Scala, functional programming, and related technologies. Contribute...

We are seeking a big data engineer with Java experience to join our modernization initiative team. The candidate will be responsible for implementing and maintaining high volume online and offline batch and transactional client facing applications in cloud and On-prem. Candidate will also responsible for the creation and maintenance of our Code & database infrastructure, including collecting and maintaining data, ensuring the integrity of our data, and training data models. Responsibilities: * Design and implement efficient data handling system based on Java and Spark. * Perform and oversee tasks such as programming, writing scripts, calling APIs, and writing complex SQL queries. * Implement...

Data Architect is responsible to define and lead the Data Architecture, Data Quality, Data Governance, ingesting, processing, and storing millions of rows of data per day. This hands-on role helps solve real big data problems. You will be working with our product, business, engineering stakeholders, understanding our current eco-systems, and then building consensus to designing solutions, writing codes and automation, defining standards, establishing best practices across the company and building world-class data solutions and applications that power crucial business decisions throughout the organization. We are looking for an open-minded, structured thinker passionate about building systems...

About Koantek: Koantek is a Databricks Pure-Play Elite Partner, helping enterprises modernize faster and unlock the full power of Data and AI. Backed by Databricks Ventures and honored as a six- As time Databricks Partner of the Year, we enable global enterprises to modernize at speed, operationalize AI, and realize the full value of their data. Our deep expertise spans industries such as healthcare, financial services, retail, and SaaS, delivering end-to-end solutions from rapid prototyping to production-scale AI deployments. We deliver tailored solutions that enable businesses to leverage data for growth and innovation. Our team of experts utilizes deep industry knowledge combined with cut...

About the role PubMatic is seeking Data Analytics focused Senior Software Engineers with expertise in building and optimizing AI agents, including strong skills in Hadoop, Spark, Scala, Kafka, Spark Streaming, and cloud-based solutions, with proficiency in programming languages such as Scala and Python. You will be responsible for developing advanced AI agents that enhance our data analytics capabilities, enabling our platform to handle complex information retrieval, contextual understanding, and adaptive interactions, ultimately improving our data-driven insights and analytical workflows What You’ll Do Build, design, and implement our highly scalable, fault-tolerant big data platform to pro...

Role Overview: At PwC, you will be part of the managed services team focusing on outsourced solutions and providing support to clients across various functions. Your role will involve helping organizations streamline operations, reduce costs, and enhance efficiency by managing key processes and functions. You will leverage your skills in project management, technology, and process optimization to deliver high-quality services to clients. Specifically, as a Data Engineer Offshore, you will play a crucial role in designing, implementing, and maintaining scalable data pipelines and systems to support data-driven initiatives. Key Responsibilities: - Design, develop, and maintain scalable ETL pip...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

JLL is a leading professional services firm that specializes in real estate and investment management. Our vision is to reimagine the world of real estate, creating rewarding opportunities and amazing spaces where people can achieve their ambitions. In doing so, we will build a better tomorrow for our clients, our people and our communities. JLL Technologies is a specialized group within JLL. At JLL Technologies, our mission is to bring technology innovation to commercial real estate. We deliver unparalleled digital advisory, implementation, and services solutions to organizations globally. Our goal is to leverage technology to increase the value and liquidity of the world's buildings, while...

We are looking for a Senior Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Data Engineer 2 at JLL Technologies, you will: Contributes to the design of information infrastructure, and data management processes to...

We are looking for a Senior Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Data Engineer 2 at JLL Technologies, you will: Contributes to the design of information infrastructure, and data management processes to...

JLL is a leading professional services firm that specializes in real estate and investment management. Our vision is to reimagine the world of real estate, creating rewarding opportunities and amazing spaces where people can achieve their ambitions. In doing so, we will build a better tomorrow for our clients, our people and our communities. JLL Technologies is a specialized group within JLL. At JLL Technologies, our mission is to bring technology innovation to commercial real estate. We deliver unparalleled digital advisory, implementation, and services solutions to organizations globally. Our goal is to leverage technology to increase the value and liquidity of the world's buildings, while...

We are looking for a Senior Data Engineer who is self-starter to work in a diverse and fast-paced environment that can join our Enterprise Data team. This is an individual contributor role that is responsible for designing and developing of data solutions that are strategic for the business and built on the latest technologies and patterns. This a global role that requires partnering with the broader JLLT team at the country, regional and global level by utilizing in-depth knowledge of data, infrastructure, technologies and data engineering experience. As a Data Engineer 2 at JLL Technologies, you will: Contributes to the design of information infrastructure, and data management processes to...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...