KAFKA ARCHITECT

Job Summary:

Key Responsibilities

• Manage and maintain Kafka clusters in non prod and prod environments
• Set up and configure Kafka clusters with Zookeeper or Kraft
• Implement and manage various Kafka connectors
• Develop and maintain ksqlDB streams and tables
• Handle Kafka operations on Kubernetes
• Create and manage Docker images when not available in public registries
• Write and manage YAML files for configuration
• Implement CICD pipelines using GitLab
• Monitor Kafka environments using tools like Datadog
• Automate infrastructure management using Terraform
• Manage AWS services including security groups load balancers Route 53 networking S3 and ECR
• Troubleshoot and resolve Kafka and infrastructure related issues
• Adhere to incident change and problem management processes

Skills and Abilities

• Strong knowledge of Kafka architecture
• Handson experience with Kafka cluster setup using Zookeeper or Kraft
• Proficiency in working with Kafka connectors and ksqlDBFlink
• Experience with Kubernetes for Kafka deployment and management
• Ability to create Docker images and work with containerized environments
• Proficiency in writing YAML configurations
• Strong troubleshooting and debugging skills
• Experience with monitoring tools like Datadog
• Familiarity with GitLab CICD pipelines
• Proficiency in Terraform for infrastructure automation
• Strong understanding of AWS cloud services
• Experience with IT service management processes incident change and problem management

Experience

• 10 years of experience managing Kafka clusters in production environments
• Experience setting up and maintaining Kafka on Kubernetes
• Handson experience with AWS cloud infrastructure
• Experience with CICD pipelines infrastructure automation and monitoring tools
• Certifications confluent AWS