Kafka Migration Developer

Job Summary:

• Kafka & Migration Developer to help with an ongoing migration from Splunk to Grafana.
• Strong Kafka experience - understand architecture
• Grafana experience - Splunk nice to have but not necessary since they are moving away from it
• Experience building/managing dashboards
• Deployments done through terraform (more critical skillset)
• Hands on python & Kubernetes experience

Core Skillets Required (Must have):

• Kafka:

  Expertise in Kafka architecture, producers, consumers, topics, partitions, and Kafka Streams.

• Grafana:

  Experience building and managing dashboards, panel customization, and alerting in Grafana.

• Splunk:

  Strong knowledge of Splunk for log ingestion, querying (SPL), and dashboard creation.

• Terraform

  : Infrastructure as Code (IaC) for provisioning and managing Kafka clusters, Grafana, and other resources.

• Python:

  Proficient in Python scripting for automation, data processing, and integration tasks.

Recommended Additional Skills & Tools:

• Kafka Ecosystem: Kafka Connect, Kafka Streams or ksqlDB, Schema Registry, Confluent Platform
• Observability & Monitoring: Prometheus (metrics collection), OpenTelemetry (distributed tracing), ElasticSearch/Logstash/Beats (log processing), Alertmanager, Grafana Alerting
• Cloud & Infrastructure: Kubernetes (deployment & orchestration), Helm (package management), AWS/Azure/GCP cloud services related to Kafka and monitoring
• Dashboard & Visualization: Grafana Loki (log aggregation), JSON/YAML for dashboard templating, API usage for automation
• Data Processing & Scripting: Advanced Python libraries (e.g., Pandas), Bash/PowerShell scripting for automation
• Security & Compliance: Kafka security (SSL, SASL, ACLs), securing data pipelines and observability infrastructure
• Version Control & CI/CD: Git workflows, Jenkins/GitLab CI/GitHub Actions for automated deployment pipelines
• Performance & Troubleshooting: Kafka performance tuning, troubleshooting, Grafana dashboard optimization

Additional Notes:

• Experience with containerization and orchestration (Docker, Kubernetes) is highly valuable for deploying scalable Kafka and Grafana environments.
• Familiarity with monitoring frameworks and distributed tracing will improve observability during migration.
• Strong automation skills help reduce manual effort in dashboard migration and infrastructure provisioning.
• Understanding security best practices ensures data integrity and access control across Kafka and observability stacks.