Grafana Stack Consultant - CI/CD Pipeline

As a highly skilled Grafana Stack Consultant, your role will involve managing and scaling self-hosted observability platforms. This full-time contract opportunity allows you to work remotely from anywhere. Your expertise will be crucial in analyzing, troubleshooting, and optimizing the existing observability infrastructure, which includes Grafana, Loki, Tempo, and Mimir operating in a Docker Swarm environment across multiple Linux VMs. The system ingests approximately 500GB of logs per day using Wasabi S3 as the object storage backend. Your key responsibilities will include: - Conducting in-depth performance analysis of the Grafana observability stack, focusing on Loki, Tempo, and Mimir - Identifying and remedying performance bottlenecks in log ingestion, indexing, and query execution - Optimizing system configurations for enhanced throughput, latency, and fault tolerance - Troubleshooting and resolving instability and downtime in the observability pipeline - Recommending and implementing best practices for operating the Grafana stack in Docker Swarm environments - Collaborating with internal engineering teams to guide architecture improvements and ensure alignment with reliability goals Qualifications required: - 5 years of hands-on experience with self-hosted Grafana, Loki, Tempo, and Mimir - Strong understanding of observability architectures, including metrics, logs, and tracing - Proven track record of operating these components at scale, particularly in high-ingestion environments - Proficiency in container orchestration, especially Docker Swarm, and familiarity with Linux VM-based deployments - Experience with integrating and tuning object storage backends like Wasabi S3 or other S3-compatible services - Ability to diagnose complex issues across distributed systems and propose effective solutions - Excellent problem-solving skills and ability to work independently in a remote setting Additional Company Details: The company is focused on reliability goals and values collaboration with internal teams for architecture improvements. (Note: Additional details about the company were not provided in the job description.),