Job
Description
A pioneer and innovator in its industry, Qualitest has been recognized in the highest Leader position in Everest Groups Quality Engineering Services for Mid-market Enterprises PEAK Matrix Assessment 2024 report and has also been recognized as a Leader in The Forrester Wave : Continuous Automation and Testing Services Q2 2024 report. Qualitest has offices in the United States, United Kingdom, Israel, Romania, India, Mexico, Portugal, Switzerland, and Argentina. Job Description: Model Evaluation Testing: Design and execute comprehensive test cases to evaluate generative AI models for accuracy, relevance, bias, and robustness. Prompt Response Analysis: Create structured prompts and analyze AI-generated responses to ensure alignment with business objectives and ethical standards. Automation Benchmarking: Develop and implement automated testing frameworks using tools like Ragas, DeepEval, and LangTest to establish benchmark metrics and enhance model reliability. Edge Case Adversarial Testing: Identify and test edge cases, adversarial inputs, and failure scenarios to strengthen model robustness and improve safety. Collaboration Reporting: Partner with AIML developers to report findings, recommend improvements, and refine model training and evaluation processes. 3 must haves for AI tester - Understanding of AI/ML Concepts , Generative AI (Gen AI) Concepts, LLM Models - Critical Thinking Ethical Awareness - Ability to analyze AI outputs critically, identify biases, edge cases, and ethical concerns, and ensure fairness, reliability, and accuracy in AI-driven decisions. - Knowledge of Automation framesworks and manual testing
