210 Spark Streaming Jobs - Page 9

JOB DESCRIPTION Collaborate with customer to gather requirements and to understand their business processes. Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, SQL and Azure or AWS data technologies. Build analytics tools that utilize the data pipeline to provide actionable insigh...

About the Role Were seeking a highly skilled Data Engineer with a strong development background and a passion for transforming data into valuable insights. The ideal candidate will play a key role in designing, building, and maintaining scalable data pipelines and analytics solutions that support critical business decisions. What youll be doing Design, build, and optimize robust data pipelines for large-scale data processing Write complex SQL queries for data extraction, transformation, and reporting Collaborate with analytics and reporting teams to deliver data-driven insights Develop scalable solutions using programming languages such as Java, Python, and Node.js Integrate APIs and third-p...

Strong problem-solving skills with a focus on product development. Domain expertise in Big Data, Data Platforms, and Distributed Systems. Proficiency in Java, Scala, or Python (hands-on experience with Apache Spark is essential). Experience with data ingestion frameworks such as Apache Storm, Flink, or Spark Streaming. Experience with streaming technologies like Kafka, Kinesis, Oplogs, Binlogs, or Debezium. Strong database skills with experience in HDFS, Delta Lake, Iceberg, or Lakehouse architectures.

Job Summary Synechron is seeking an experienced Big Data Developer with strong expertise in Spark, Scala, and Python to lead and contribute to large-scale data projects. The role involves designing, developing, and implementing robust data solutions that leverage emerging technologies to enhance business insights and operational efficiency. The successful candidate will play a key role in driving innovation, mentoring team members, and ensuring the delivery of high-quality data products aligned with organizational objectives. Software Requirements Required: Apache Spark (latest stable version) Scala (version 2.12 or higher) Python (version 3.6 or higher) Big Data tools and frameworks support...

Strong programming experience in Scala . Hands-on experience in Apache Spark using Scala , with deep understanding of distributed computing concepts. Experience with Core Spark APIs and Spark Streaming APIs . Familiarity with HDFS, Hive, S3, MongoDB , and relational databases. Experience in creating Scala/Spark jobs for large-scale data processing. Sound exposure to ETL pipeline implementation, batch scheduling, and automation . Proficiency in Hive queries . Understanding of big data concepts and distributed systems . Experience with Unix commands and shell scripting . Familiarity with Java (basic and advanced) is preferred.

Minimum 4+ years of development and design experience in Java/Scala with Flink, Beam (or spark streaming) and Kafka Extensive coding experience and knowledge in Event driven and streaming architecture Experience in JVM tuning for performance Knowledge on Containerisation using Docker and Kubernetes. Working knowledge on Caching systems, with particular experience using Redis will be preferable. [Nice to have] Linux OS configuration and use, including shell scripting. Good hands on experience with design patterns and their implementation. Well versed with CI/CD principles (GitHub, Jenkins etc.) , and actively involved in solving, troubleshooting issues in distributed services ecosystem Experi...

Role Purpose The purpose of this role is to interpret data and turn into information (reports, dashboards, interactive visualizations etc) which can offer ways to improve a business, thus affecting business decisions. Big Data JD: Skills and Qualifications: 6+ years of experience Good Understanding of Bigdata and Spark ecosystem Proficiency in Apache Spark development with Scala Experience in building Spark Streaming and Batch applications Strong experience in creating applications for Big Data and other Data & Analytics technologies (Spark, Scala, Hadoop, Hive etc.) Ability to analyze Spark job performance in Spark UI and apply required performance tuning Experience of working with Linux Sc...

About the Role: This position requires someone to work on complex technical projects and closely work with peers in an innovative and fast-paced environment. For this role, we require someone with a strong product design sense & specialized in Hadoop and Spark technologies. Requirements: Minimum 6-8 years of experience in Big Data technologies. The position Grow our analytics capabilities with faster, more reliable tools, handling petabytes of data every day. Brainstorm and create new platforms that can help in our quest to make available to cluster users in all shapes and forms, with low latency and horizontal scalability. Make changes to our diagnosing any problems across the entire techni...

What you’ll do Work with business stakeholders to understand their business needs. Create data pipelines that extract, transform, and load (ETL) from various sources into a usable format in a Data warehouse. Clean, filter, and validate data to ensure it meets quality and format standards. Develop data model objects (tables, views) to transform the data into unified format for downstream consumption. Expert in monitoring, controlling, configuring, and maintaining processes in cloud data platform. Optimize data pipelines and data storage for performance and efficiency. Participate in code reviews and provide meaningful feedback to other team members. Provide technical support and troubleshoot ...

Job Title:Data Engineer Experience: 5-7Years Location:Pune Job Description : Roles & Responsibilities: Create and maintain optimal data pipeline architecture Build data pipelines that transform raw, unstructured data into formats that data analyst can use to for analysis Assemble large, complex data sets that meet functional / non-functional business requirements Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and delivery of data from a wide variety of data sources using SQL and AWS Big Data ...