13 Kafka Mq Jobs

Job Description: Standing up and administer on premise Kafka cluster. Ability to architect and create reference architecture for kafka Implementation standards Provide expertise in Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control center. Ensure optimum performance, high availability and stability of solutions. Create topics, setup redundancy cluster, deploy monitoring tools, alerts and has good knowledge of best practices. Create stubs for producers, consumers and consumer groups for helping onboard applications from different languages/platforms. Provide administration and operations of the Kafka platform like provisioning, access lists Kerberos and SSL configurations. Use automation tools like provisioning using Docker, Jenkins and GitLab. Ability to perform data related benchmarking, performance analysis and tuning. Strong skills in In-memory applications, Database Design, Data Integration. Involve in design and capacity review meetings to provide suggestion in Kafka usage. Solid knowledge of monitoring tools and fine tuning alerts on Splunk, Prometheus, Grafana ,Splunk. Setting up security on Kafka. Providing naming conventions, Backup & Recovery and problem determination strategies for the projects. Monitor, prevent and troubleshoot security related issues. Provide strategic vision in engineering solutions that touch the messaging queue aspect of the infrastructure QUALIFICATIONS Demonstrated proficiency and experience in design, implementation, monitoring, and troubleshooting Kafka messaging infrastructure. Hands on experience on recovery in Kafka. 2 or more years of experience in developing/customizing messaging related monitoring tools/utilities. Good Scripting knowledge/experience with one or more (ex. Chef, Ansible, Terraform). Good programming knowledge/experience with one or more languages (ex. Java, node.js, python) Considerable experience in implementing Kerberos Security. Support 24*7 Model and be available to support rotational on-call work Competent working in one or more environments highly integrated with an operating system. Experience implementing and administering/managing technical solutions in major, large-scale system implementations. High critical thinking skills to evaluate alternatives and present solutions that are consistent with business objectives and strategy. Ability to manage tasks independently and take ownership of responsibilities Ability to learn from mistakes and apply constructive feedback to improve performance Ability to adapt to a rapidly changing environment. Proven leadership abilities including effective knowledge sharing, conflict resolution, facilitation of open discussions, fairness and displaying appropriate levels of assertiveness. Ability to communicate highly complex technical information clearly and articulately for all levels and audiences. Willingness to learn new technologies/tool and train your peers. Proven track record to automate.

System Engineering & Development Design, develop, and maintain backend services and infrastructure components. Build and manage containerized applications using Docker and orchestrate them with Kubernetes, OpenShift, or similar platforms. Develop automation scripts and tools for system provisioning, configuration, and deployment. Messaging & Integration Architect and implement message-driven systems using technologies like HIVEMQ , Kafka, RabbitMQ, Azure Service Bus, or similar. Ensure reliable and scalable message processing across distributed systems. Integrate messaging solutions with microservices and backend systems. Linux Systems Administration Manage and optimize Red Hat Enterprise Linux (RHEL) environments. Perform system monitoring, performance tuning, and troubleshooting. Implement security best practices and ensure compliance with organizational policies. Collaboration & Support Work closely with DevOps, QA, and development teams to deliver high-quality solutions. Participate in code reviews, technical discussions, and architectural planning. Provide mentorship and guidance to junior engineers. Required Qualifications: Bachelors or Master’s degree in Computer Science, Engineering, or related field. 8+ years of experience in software engineering or systems development. Strong proficiency in Red Hat Linux system administration and shell scripting. Hands-on experience with messaging systems (HIVEMQ, Kafka, RabbitMQ, etc.). Solid understanding of containerization and orchestration tools (Docker, Kubernetes, OpenShift). Familiarity with CI/CD pipelines, Git, and DevOps practices. Experience with monitoring and logging tools (e.g., Prometheus, Grafana, ELK Stack). Strong analytical and problem-solving skills. Good to have Qualifications: Red Hat Certified Engineer (RHCE) or equivalent certification. Experience with cloud platforms (AWS, Azure, GCP). Knowledge of infrastructure as code (Terraform, Ansible). Exposure to microservices architecture and distributed systems

5+ yrs in Python, Django Microservices Architecture and API development Deploy via Kubernetes Redis for performance tuning Celery for distributed task DBs: PostgreSQL, Time-series Redis, Celery, RabbitMQ/Kafka Microservices architecture exp

Job Title: Confluent CDC System Analyst Role Overview: A leading bank in the UAE is seeking an experienced Confluent Change Data Capture (CDC) System Analyst/ Tech lead to implement real-time data streaming solutions. The role involves implementing robust CDC frameworks using Confluent Kafka , ensuring seamless data integration between core banking systems and analytics platforms. The ideal candidate will have deep expertise in event-driven architectures, CDC technologies, and cloud-based data solutions . Key Responsibilities: Implement Confluent Kafka-based CDC solutions to support real-time data movement across banking systems. Implement event-driven and microservices-based data solutions for enhanced scalability, resilience, and performance . Integrate CDC pipelines with core banking applications, databases, and enterprise systems . Ensure data consistency, integrity, and security , adhering to banking compliance standards (e.g., GDPR, PCI-DSS). Lead the adoption of Kafka Connect, Kafka Streams, and Schema Registry for real-time data processing. Optimize data replication, transformation, and enrichment using CDC tools like Debezium, GoldenGate, or Qlik Replicate . Collaborate with Infra team, data engineers, DevOps teams, and business stakeholders to align data streaming capabilities with business objectives. Provide technical leadership in troubleshooting, performance tuning, and capacity planning for CDC architectures. Stay updated with emerging technologies and drive innovation in real-time banking data solutions . Required Skills & Qualifications: Extensive experience in Confluent Kafka and Change Data Capture (CDC) solutions . Strong expertise in Kafka Connect, Kafka Streams, and Schema Registry . Hands-on experience with CDC tools such as Debezium, Oracle GoldenGate, or Qlik Replicate . Hands on experience on IBM Analytics Solid understanding of core banking systems, transactional databases, and financial data flows . Knowledge of cloud-based Kafka implementations (AWS MSK, Azure Event Hubs, or Confluent Cloud) . Proficiency in SQL and NoSQL databases (e.g., Oracle, MySQL, PostgreSQL, MongoDB) with CDC configurations. Strong experience in event-driven architectures, microservices, and API integrations . Familiarity with security protocols, compliance, and data governance in banking environments. Excellent problem-solving, leadership, and stakeholder communication skills .

Lead the solution design and implementation of core platform features API Design and Implementation Provide Operational support with building platform monitoring tools/dashboards, ad hoc reports Defect Fixes, Performance Testing, Endurance testing Willingness to work in second shift from offshore so that he can overlap with Onsite team Qualifications Overall 8+ years of developing internet-scale solution development primarily using Java, Spring Boot and no-sql databases Must have demonstrated proficiency and experience in the following tools and technologies Java 11 (Lambdas, Streams, Completable Future, optional, generics) Spring boot (webflux , Reactor 3), spring-data, REST Java functional and reactive programming Test Driven Development Asynchronous Reactive Micro services utilizing Vert.x REST APIs using Spring Boot 2.0 (reactive) and skilled in Open API (swagger) specification Designing database schemas, index design, optimizations for query tuning Working knowledge of cloud technologies (eg. docker, kubernetes, jager, prometheus) Modern software engineering toolsgit workflows, gradle, load testing tools, mock frameworks Good knowledge of messaging systems like Kafka, mq Take pride in writing good clean code, perform peer code reviews and architecture reviews. A bachelors degree in engineering or related field Java certification is a plus Candidate Soft Skills Demonstrated evidence to learn new skills Demonstrated evidence of going above and beyond to make projects successful Good Communication is necessary. skills Core Java SpringBoot Rest API VertX No SQL Kafka MQ

: Lead the solution design and implementation of core platform features API Design and Implementation Provide Operational support with building platform monitoring tools/dashboards, ad hoc reports Defect Fixes, Performance Testing, Endurance testing Willingness to work in second shift from offshore so that he can overlap with Onsite team Qualifications Overall 4+ years of developing internet-scale solution development primarily using Java, Spring Boot and no-sql databases Must have demonstrated proficiency and experience in the following tools and technologies Java 11 (Lambdas, Streams, Completable Future, optional, generics) Spring boot (webflux , Reactor 3), spring-data, REST Java functional and reactive programming Test Driven Development Asynchronous Reactive Micro services utilizing Vert.x REST APIs using Spring Boot 2.0 (reactive) and skilled in Open API (swagger) specification Designing database schemas, index design, optimizations for query tuning Working knowledge of cloud technologies (eg. docker, kubernetes, jager, prometheus) Modern software engineering toolsgit workflows, gradle, load testing tools, mock frameworks Good knowledge of messaging systems like Kafka, mq Take pride in writing good clean code, perform peer code reviews and architecture reviews. A bachelors degree in engineering or related field Java certification is a plus Candidate Soft Skills Demonstrated evidence to learn new skills Demonstrated evidence of going above and beyond to make projects successful Good Communication is necessary. skills Core Java SpringBoot Rest API VertX No SQL Kafka MQ

Description: Must have strong communication and project management experience with a developer/ tech background Must have Java and API development experience Ability to effectively interpret Technical and Business objectives as well as challenges and support the team in delivering Ability to drive projects end2 end, work with a diverse set of teams and stakeholders 3+ years of Software Project Management experience Must have Java and API development experience Ability to effectively interpret Technical and Business objectives as well as challenges and support the team in delivering Ability to drive projects end-2-end, work with a diverse set of teams Willingness to learn new technologies, experiment (write code when needed) and exploit them to their optimal potential Experience in clarifying objectives, dealing with ambiguity/under-defined problems, advocating for simplification, and influencing teams Excellent oral and written communication skills; Ability to effectively communicate across third parties and technical and business product managers. Ability to get industry leading solutions to market efficiently and effectively Influence team members with innovative changes and improvements by challenging status-quo and demonstrating risk taking. Must have strong communication and project management experience with a developer/ tech background Must have Java and API development experience Ability to effectively interpret Technical and Business objectives as well as challenges and support the team in delivering Ability to drive projects end-2-end, work with a diverse set of teams and stakeholders 3+ years of Software Project Management experience Must have Java and API development experience Ability to effectively interpret Technical and Business objectives as well as challenges and support the team in delivering Ability to drive projects end-2-end, work with a diverse set of teams Willingness to learn new technologies, experiment (write code when needed) and exploit them to their optimal potential Experience in clarifying objectives, dealing with ambiguity/under-defined problems, advocating for simplification, and influencing teams Excellent oral and written communication skills; Ability to effectively communicate across third parties and technical and business product managers. Ability to get industry leading solutions to market efficiently and effectively Influence team members with innovative changes and improvements by challenging status-quo and demonstrating risk taking. Drive scrum team ceremonies, PI planning, reduce risks, etc. Publish commitments of planning sessions (whether sprint or PI planning sessions) Monthly Product & Technology sync with Portfolio Manager Collaborate with Product & Technology to identify top features per team Trouble-shoot issues and surface up impediments with delivery Proficiently manage JIRA spacesassume the position as project lead/admin, ensure consistency in set-up and usage. Core Java API Development Project Management SAFe Kafka MQ

Excellent knowledge of C# with Multi-threading, Async. Good command over OOP concepts, SOLID principles, Design patterns. Experience in REST API/Microservices using ASP.Net WebAPI. Only ASP.Net MVC is not acceptable . Securing web applications using Authentication Tokens, Certificates, OAuth etc. Caching, Distributed caching. Pub/Sub, Queues/Topics Message Broker, Any one Queuing system. SQL Server knowledge, Joins, Stored Procedures, Functions, writing complex queries. SSIS/SSRS Experience on Lambda, CloudWatch, API Gateway, S3 Bucket, EC2, SNS, SQS, ELB, Docker/Kubernetes & Kafka MQ, IAM, Authorization, and Access control, SaaS etc.

Develop and maintain Kafka-based data pipelines for real-time processing. Implement Kafka producer and consumer applications for efficient data flow. Optimize Kafka clusters for performance, scalability, and reliability. Design and manage Grafana dashboards for monitoring Kafka metrics. Integrate Grafana with Elasticsearch, or other data sources. Set up alerting mechanisms in Grafana for Kafka system health monitoring. Collaborate with DevOps, data engineers, and software teams. Ensure security and compliance in Kafka and Grafana implementations. Requirements: 8+ years of experience in configuring Kafka, ElasticSearch and Grafana Strong understanding of Apache Kafka architecture and Grafana visualization. Proficiency in .Net, or Python for Kafka development. Experience with distributed systems and message-oriented middleware. Knowledge of time-series databases and monitoring tools. Familiarity with data serialization formats like JSON. Expertise in Azure platforms and Kafka monitoring tools. Good problem-solving and communication skills. Mandate : Create the Kafka dashboards , Python/.NET Note: Candidate must be immediate joiner.

Kafka/MSK Linux In-depth understanding of Kafka broker configurations, zookeepers, and connectors Understand Kafka topic design and creation. Good knowledge in replication and high availability for Kafka system ElasticSearch/OpenSearch Perks and benefits PF, ANNUAL BONUS, HEALTH INSURANCE

Design & develop full-stack Java applications using Spring ecosystem Build RESTful Web Services&Microservices architecture Develop responsive UIs using HTML5, React/Angular, JavaScript, CSS Implement integration solutions using REST APIs and Kafka/MQ

Excellent knowledge of C# with Multi-threading, Async. Good command over OOP concepts, SOLID principles, Design patterns. Experience in REST API/Microservices using ASP.Net WebAPI. Only ASP.Net MVC is not acceptable . Securing web applications using Authentication Tokens, Certificates, OAuth etc. Caching, Distributed caching. Pub/Sub, Queues/Topics Message Broker, Any one Queuing system. SQL Server knowledge, Joins, Stored Procedures, Functions, writing complex queries. SSIS/SSRS Experience on Lambda, CloudWatch, API Gateway, S3 Bucket, EC2, SNS, SQS, ELB, Docker/Kubernetes & Kafka MQ, IAM, Authorization, and Access control, SaaS etc.

Role & responsibilities Java, Restful API using Spring Boot, Microservices & event-based architecture, Kafka & MQ, Oracle & PostGreSQLDB Responsilblities: Lead a team, Implements Reusable components/services, Working with API Gateways, Unit Test for RESTful services, Create DDL/DML scripts.