Job
Description
Develop Kafka-based Solutions : Design and implement scalable, fault-tolerant Kafka producers and consumers in Java. Real-time Data Processing : Work with Kafka Streams or other stream-processing tools to process large volumes of real-time data efficiently. Kafka Integration : Integrate Kafka with other systems and services (such as databases, data warehouses, microservices, etc. ) for seamless data flow. Optimize Kafka Performance : Tune Kafka clusters, consumer groups, and producers for optimal throughput and fault tolerance. Data Modeling : Collaborate with data engineers to create data models and structures that best fit Kafka s architecture and improve data ingestion and retrieval. Monitoring and Troubleshooting : Set up and maintain monitoring tools to track Kafka consumer/producer performance, troubleshoot Kafka-related issues, and ensure the system s uptime. Collaborate with Cross-functional Teams : Work closely with backend developers, DevOps engineers, and data scientists to ensure Kafka pipelines align with the company s data architecture. Documentation : Document Kafka-related processes, designs, and API interfaces to help internal teams understand and maintain the system. Skills and Qualifications: Strong Java Development Skills : Expertise in Java 8 or later versions. Kafka Experience : Practical experience with Apache Kafka, including setting up, configuring, and maintaining Kafka brokers, creating Kafka producers/consumers, and working with Kafka Streams. Distributed Systems Knowledge : Understanding of distributed architectures and concepts such as fault tolerance, scalability, and load balancing. Stream Processing : Experience with stream processing frameworks like Kafka Streams, Apache Flink, or similar technologies is a plus. Message Queue Systems : Knowledge of message queue systems and protocols (e. g. , Kafka, RabbitMQ, or similar). Microservices : Experience with microservices architecture and how Kafka fits within microservices-based systems. Database Integration : Experience with integrating Kafka with relational (MySQL, PostgreSQL) or NoSQL databases (MongoDB, Cassandra). Performance Tuning : Ability to optimize and fine-tune Kafka producers, consumers, and Kafka clusters. Tools : Familiarity with tools like Zookeeper , KSQL , Confluent Platform , and Schema Registry is a plus. CI/CD & Version Control : Experience with Git, Jenkins, and building automated deployment pipelines. Kafka, Flink, Java, Micros Service, Apache, Spring & Spring Boot
