Senior SRE - IT AI Platforms

The IT AI Application Platform team is seeking a Senior Site Reliability Engineer (SRE) to develop, scale, and operate our AI Application Platform based on Red Hat technologies, including OpenShift AI (RHOAI) and Red Hat Enterprise Linux AI (RHEL AI). As an SRE you will contribute to running core AI services at scale by enabling customer self-service, making our monitoring system more sustainable, and eliminating toil through automation.

On the IT AI Application Platform team, you will have the opportunity to influence the complex challenges of scale which are unique to Red Hat IT managed AI platform services, while using your skills in coding, operations, and large-scale distributed system design. We develop, deploy, and maintain Red Hats next-generation Ai application deployment environment for custom applications and services across a range of hybrid cloud infrastructures.  We are a global team operating on-premise and in the public cloud, using the latest technologies from Red Hat and beyond.

Red Hat relies on teamwork and openness for its success. We are a global team and strive to cultivate a transparent environment that makes room for different voices. We learn from our failures in a blameless environment to support the continuous improvement of the team. At Red Hat, your individual contributions have more visibility than most large companies, and visibility means career opportunities and growth.

What you will do

The day-to-day responsibilities of an SRE involve working with live systems and coding automation. As an SRE you will be expected to:

• Build and manage our large scale infrastructure and platform services, including public cloud, private cloud, and datacenter-based
• Automate cloud infrastructure through use of technologies (e.g. auto scaling, load balancing, etc.), scripting (bash, python and golang), monitoring and alerting solutions (e.g. Splunk, Splunk IM, Prometheus, Grafana, Catchpoint etc)
• Design, develop, and become expert in AI capabilities leveraging emerging industry standards
• Participate in the design and development of software like Kubernetes operators, webhooks, cli-tools...
• Implement and maintain intelligent infrastructure and application monitoring designed to enable application engineering teams
• Ensure the production environment is operating in accordance with established procedures and best practices
• Provide escalation support for high severity and critical platform-impacting events
• Provide feedback around bugs and feature improvements to the various Red Hat Product Engineering teams
• Contribute software tests and participate in peer review to increase the quality of our codebase
• Help and develop peers capabilities through knowledge sharing, mentoring, and collaboration
• Participate in a regular on-call schedule, supporting the operation needs of our tenants
• Practice sustainable incident response and blameless postmortems
• Work within a small agile team to develop and improve SRE methodologies, support your peers, plan and self-improve

What you will bring

A bachelor's degree in Computer Science or a related technical field involving software or systems engineering is required. However, hands-on experience that demonstrates your ability and interest in Site Reliability Engineering are valuable to us, and may be considered in lieu of degree requirements. You must have some experience programming in at least one of these languages: Python, Golang, Java, C, C++ or another object-oriented language. You must have experience working with public clouds such as AWS, GCP, or Azure. You must also have the ability to collaboratively troubleshoot and solve problems in a team setting.

As an SRE you will be most successful if you have some experience troubleshooting an as-a-service offering (SaaS, PaaS, etc.) and some experience working with complex distributed systems. We like to see a demonstrated ability to debug, optimize code and automate routine tasks. We are Red Hat, so you need a basic understanding of Unix/Linux operating systems.

Desired skills

• 3+ years of experience of using cloud providers and technologies (Google, Azure, Amazon, OpenStack etc)
• 1+ years of experience administering a kubernetes based production environment
• 2+ years of experience with enterprise systems monitoring
• 2+ years of experience with enterprise configuration management software like Ansible by Red Hat, Puppet, or Chef
• 2+ years of experience programming with at least one object-oriented language; Golang, Java, or Python are preferred
• 2+ years of experience delivering a hosted service
• Demonstrated ability to quickly and accurately troubleshoot system issues
• Solid understanding of standard TCP/IP networking and common protocols like DNS and HTTP
• Demonstrated comfort with collaboration, open communication and reaching across functional boundaries
• Passion for understanding users needs and delivering outstanding user experiences
• Independent problem-solving and self-direction
• Works well alone and as part of a global team
• Experience working with Agile development methodologies