Job
Description
Job Title: Senior AI Automation Exp : 5 Years to 10 Years Location : Remote. About the Company: Galileo is the leading platform for Gen AI evaluation and observability, with a mission to democratize building safe, reliable and robust applications in the new era of AI powered software development. Our foundation is built on pioneering the early technology behind the world's most ubiquitous AI applications including Apple's Siri and Google Speech. We firmly believe that AI developers require meticulously crafted, research-driven tools to create trustworthy and high-quality generative AI applications that will revolutionize our work and lifestyle. Galileo addresses the complexities inherent in implementing, evaluating, and monitoring GenAI applications, optimizing the development process for both individual developers and teams by offering a comprehensive platform that spans the full AI development lifecycle. Galileo bridges critical gaps, significantly enhancing developers' ability to refine and deploy reliable and precise GenAI applications. Since its inception, Galileo has rapidly gained traction, serving Fortune 100 banks, Fortune 50 telecom companies, as well as AI teams at prominent organizations such as Reddit and Headspace Health, among dozens of others. Galileo has AI research at its core, with the founders coming from Google and Uber where they solved challenging AI/ML problems in the Speech, Evaluation and ML Infra domains. It is now a Series B business backed by tier 1 investors including Battery Ventures, Scale Venture Partners, and Databricks Ventures, with $68M in total funding. We are headquartered in San Francisco with locations such as New York and Bangalore, India forming our areas of future growth. Responsibilities Understand GenAI at a sufficient level of curiosity and usage (beyond simple applications such as using ChatGPT Such as understanding and usage of LangChain, LangGraph, CrewAI, OpenAI APIs etc. Be able to quickly understand our product hosted at https://app.galileo.ai/ Be able to write automated testing scripts for the UI layer and for various API (or Backend) layers REST, Python) and our SDKs Python and Typescript. These could be unit test cases, functional test cases or integration tests Main goal will be to increase our test coverage and hence improve the quality of us product. Can work in a fast-paced environment and can get ideas quickly off the ground with minimal support (ideally has experience working in a mix of large enterprises and startup experience) Skill Requirements Testing Frameworks like Playwright/Selenium, PyTest Programming languages: Python, Typescript Basic understanding of LLM (OpenAI/Gemini/) API and SDK Basic understanding of AI frameworks like Langchain, CrewAI etc. If you are passionate about applying your experience in the AI/ML field to an emerging and advancing field of Generative AI, this is a great opportunity for you. You will get to work on a ground breaking product poised to revolutionize the way companies approach LLM Evaluations.
