Observability Engineer (Cloud Engineer) (Otel, AWS, Grafana)

The Opportunity

"FICO is seeking a Full-Stack observability Lead Engineer to design, maintain, and optimize our observability platform. The ideal candidate will be an expert in Open telemetry(Otel) instrumentation and development as well as Grafana OSS (Loki, Mimir, Tempo). This role is highly hands-on, with responsibility for contributing building scalable observability solutions, integrating them into development pipelines and enabling data-driven observability across systems."- Director, Cloud Engineering

What You’ll Contribute

• Otel Instrumentation and development where you should design, implement, and optimize Otel instrumentation and pipelines across services to capture high-quality metrics, logs, and traces.
• Build and optimize the observability stack with Grafana OSS, Loki, Mimir, Tempo, and Otel to ensure scalability, performance, and minimal downtime.
• Integration with Development and Deployment Pipelines where you should embed observability into CI/CD pipelines and application lifecycles while driving dashboards-as-code and alerts-as-code practices.
• Build and automate dashboards, telemetry collection, alerting, and enrichment to improve visibility and reduce operational overhead.
• Collaborate across teams, engineers, promote observability best practices, and partner with DevOps, SRE, and cloud teams to align observability with infrastructure and application delivery.

What We’re Seeking

• Deep hands-on experience with OpenTelemetry instrumentation & development.
• Proven experience with Grafana OSS stack –Loki, Mimir, Tempo.
• Well versed with various serviced of AWS
• Ability to work with terraform
• Experience with Cribl, FluentBit, or similar data routing, aggregation tools.
• Proficiency in scripting/automation (Python, Shell, Perl).
• Advanced AWS cloud experience for observability in distributed systems.
• Familiarity with CI/CD and orchestration tools (Jenkins, Ansible, Terraform).
• Background in monitoring/observability tools such as Splunk, AppDynamics, Zabbix, ServiceNow.

Our Offer to You

• An inclusive culture strongly reflecting our core values: Act Like an Owner, Delight Our Customers and Earn the Respect of Others.
• The opportunity to make an impact and develop professionally by leveraging your unique strengths and participating in valuable learning experiences.
• Highly competitive compensation, benefits and rewards programs that encourage you to bring your best every day and be recognized for doing so.
• An engaging, people-first work environment offering work/life balance, employee resource groups, and social events to promote interaction and camaraderie.