281 Spark Streaming Jobs - Page 12

Strong programming experience in Scala . Hands-on experience in Apache Spark using Scala , with deep understanding of distributed computing concepts. Experience with Core Spark APIs and Spark Streaming APIs . Familiarity with HDFS, Hive, S3, MongoDB , and relational databases. Experience in creating Scala/Spark jobs for large-scale data processing. Sound exposure to ETL pipeline implementation, batch scheduling, and automation . Proficiency in Hive queries . Understanding of big data concepts and distributed systems . Experience with Unix commands and shell scripting . Familiarity with Java (basic and advanced) is preferred.

Minimum 4+ years of development and design experience in Java/Scala with Flink, Beam (or spark streaming) and Kafka Extensive coding experience and knowledge in Event driven and streaming architecture Experience in JVM tuning for performance Knowledge on Containerisation using Docker and Kubernetes. Working knowledge on Caching systems, with particular experience using Redis will be preferable. [Nice to have] Linux OS configuration and use, including shell scripting. Good hands on experience with design patterns and their implementation. Well versed with CI/CD principles (GitHub, Jenkins etc.) , and actively involved in solving, troubleshooting issues in distributed services ecosystem Experi...

Role Purpose The purpose of this role is to interpret data and turn into information (reports, dashboards, interactive visualizations etc) which can offer ways to improve a business, thus affecting business decisions. Big Data JD: Skills and Qualifications: 6+ years of experience Good Understanding of Bigdata and Spark ecosystem Proficiency in Apache Spark development with Scala Experience in building Spark Streaming and Batch applications Strong experience in creating applications for Big Data and other Data & Analytics technologies (Spark, Scala, Hadoop, Hive etc.) Ability to analyze Spark job performance in Spark UI and apply required performance tuning Experience of working with Linux Sc...

About the Role: This position requires someone to work on complex technical projects and closely work with peers in an innovative and fast-paced environment. For this role, we require someone with a strong product design sense & specialized in Hadoop and Spark technologies. Requirements: Minimum 6-8 years of experience in Big Data technologies. The position Grow our analytics capabilities with faster, more reliable tools, handling petabytes of data every day. Brainstorm and create new platforms that can help in our quest to make available to cluster users in all shapes and forms, with low latency and horizontal scalability. Make changes to our diagnosing any problems across the entire techni...

What you’ll do Work with business stakeholders to understand their business needs. Create data pipelines that extract, transform, and load (ETL) from various sources into a usable format in a Data warehouse. Clean, filter, and validate data to ensure it meets quality and format standards. Develop data model objects (tables, views) to transform the data into unified format for downstream consumption. Expert in monitoring, controlling, configuring, and maintaining processes in cloud data platform. Optimize data pipelines and data storage for performance and efficiency. Participate in code reviews and provide meaningful feedback to other team members. Provide technical support and troubleshoot ...

Job Title:Data Engineer Experience: 5-7Years Location:Pune Job Description : Roles & Responsibilities: Create and maintain optimal data pipeline architecture Build data pipelines that transform raw, unstructured data into formats that data analyst can use to for analysis Assemble large, complex data sets that meet functional / non-functional business requirements Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and delivery of data from a wide variety of data sources using SQL and AWS Big Data ...