AI Tester

Job Description

Understand Design and implementation of AI models using Large Language Models (LLMs) with a focus on OpenAI or Azure OpenAI or LangSmith technologies. Test multi-agent workflows and agentic frameworks to facilitate collaboration among AI agents. Utilize effective prompting techniques to enhance the performance of AI models and ensure optimal outputs. Implement Retrieval-Augmented Generation (RAG) techniques and manage vector databases to improve information retrieval processes. Testing agent tools integration using Model Context Protocol (MCP) for seamless communication and operation of AI agents. Monitor and analyze system performance through observation and telemetry to improve AI model accuracy and reliability. Collaborate with cross-functional teams to test RESTful APIs that support AI functionalities and integrations. Ensure the security of AI applications by testing implementations of security fundamentals, including OAuth2.x authentication methods. Testing SQL databases (SQL Server preferred) to ensure data integrity and accessibility. Write efficient code in Python and TypeScript, adhering to best practices and coding standards. Optionally, utilize C# for certain application components testing as needed. Mandatory Skills Proven experience working with Large Language Models (LLMs), with specific experience in OpenAI or Azure OpenAI or LangChain preferred. Strong understanding of agentic frameworks and multiagent systems. Expertise in effective prompting strategies for AI models. Familiarity with Retrieval Augmented Generation (RAG) and vector databases. Experience with agent tools, particularly Model Context Protocol (MCP).Proficient in monitoring and telemetry techniques for AI applications. Strong programming skills in Python and TypeScript experience with C# is a plus. Knowledge of REST API fundamentals and best practices. Understanding of security fundamentals, including OAuth2.x and secure coding practices. Experience with SQL databases, specifically SQL Server, is highly desirable. Strong problem-solving skills, with a focus on delivering high quality software solutions. Excellent communication and teamwork skills, with the ability to collaborate effectively across departments. Good to Have Skills TFS, GitHub tool knowledge FrameWork Design Along with technical skills, need Good Communication and articulation skills