Kafka Admin

Job Description

Qualification Perform Admin functions, Setting Kafka cluster and responsible for cluster migration Monitoring activities (Support team) and Hands on Monitoring tool Confluent Control Centre, Cloudera Manager Deployment and Management of Confluent Kafka Performance monitoring and investigation and Streams based application problem investigation Container Orchestration (Kubernetes), Container technologies (Docker) Infrastructure as code (Terraform, Concourse, Veracode) Service Monitoring and Graphing (Prometheus + Grafana, Splunk) Cluster maintenance activities all Environments Adding node and removing node on prod cluster and rebalancing DevOps Pipeline setup CI/CD for Kafka Ansible, Deployments, AZURE Data Bricks, On-premises Hadoop, Spark etc. all the data pipelines To work on Kafka Prerequisites of infra and network Confluent Kafka license configuration for the components/nodes in UAT and PROD Non Prod cluster activities and migration plan support and adhere to the standards and make as like Production Disaster recovery setup and regular failover drill runs on the cluster Develop Prerequisite scripts and automation scripts for health check and monitoring the cluster Automate monitoring and proactive alerts App Dynamics Integration from Kafka and Log Analyses (Splunk Integration for the logs) Prepare and execute load/performance test cases Automation of monitoring and ensure cluster healthy with the best practices Security configuration and protocol as per the Mashreq ISG requirements LDAP and Cluster access strategy 24x7 availability will be for Critical Production issue marked Severity 1 (or P1 in Confluent terms) only and NOT Sev2 or Sev3 Qualifications BE or degree in Computer Science or a related technical field 7+ years of experience in Confluent Kafka/Apache Kafka cluster installation and maintenance Expert admin experience in Linux and distributed cluster environments Good Experience in Ansible and DevOps for Admin activities