Lead Observability Engineer

Investcloud

8 - 12 years

25 - 30 Lacs

hubli mangaluru mysuru bengaluru belgaum

Posted:1 week ago| Platform:

Apply

Skills Required

performance tuning automation architecture gcp configuration management infrastructure splunk troubleshooting monitoring spl

Work Mode

Work from Office

Job Type

Full Time

Job Description

Key Responsibilities

Own the design, deployment, and lifecycle management of the Splunk Enterprise platform, including indexer and search head clustering, forwarders, and knowledge objects.

Define and implement best practices for data onboarding, parsing, enrichment, and storage to support observability use cases.

Collaborate with infrastructure, DevOps, security, and application teams to build reliable, scalable observability solutions.

Develop advanced SPL searches, correlation rules, alerts, and performance dashboards.

Improve alert quality and reduce noise through smarter event correlation and visualization.

Drive observability maturity initiatives including logging standardization, automation, and self-service access to telemetry data.

Evaluate and integrate additional observability and monitoring tools (e.g., Prometheus, Grafana, LogicMonitor, AppDynamics, Dynatrace, etc.) to complement existing capabilities.

Lead troubleshooting and incident response efforts where visibility and telemetry data are required.

Mentor junior engineers and influence platform and observability architecture decisions.

Qualifications

8-12 years of progressive experience in observability, infrastructure monitoring, or SRE roles.

Minimum 7 years of direct hands-on experience with Splunk Enterprise at enterprise scale.

Deep knowledge of Splunk architecture, including clustering, ingestion pipelines, search performance tuning, and index lifecycle policies.

Advanced proficiency with SPL (Search Processing Language) and dashboarding.

Experience building and scaling log pipelines using technologies such as syslog, Fluentd, Logstash, Cribl, etc.

Familiarity with cloud platforms (AWS, Azure, or GCP) and hybrid infrastructure environments.

Experience working with configuration management and infrastructure-as-code tools (e.g., Terraform, Ansible).

Excellent collaboration, problem-solving, and communication skills.

Must Have

Splunk certifications (e.g., Certified Architect, Consultant, or Admin).

Experience with APM and tracing tools (e.g., OpenTelemetry, Jaeger, New Relic, etc.).