Generative AI testing

Key Responsibilities:

• Collaborate with the development team to understand project requirements and design effective testing plans and strategies for Gen AI applications.  
• Execute various testing methodologies, including functionality, regression, performance, and usability testing to evaluate AI application performance and functionality.  
• Develop and maintain automated test scripts such as Automated Prompt Execution, Regression Testing for AI Models, API and Integration Testing.  
• Identify, document, and prioritize bugs and issues during testing using bug tracking systems; collaborate with developers to ensure timely resolution.  
• Analyze performance metrics and conduct load testing to identify bottlenecks and optimize performance in Gen AI applications.  
• Collaborate with cross-functional teams including developers, designers, and product managers to ensure alignment on quality standards and testing objectives.  
• Create detailed test plans, test cases, and test reports to communicate results and recommendations to stakeholders.  
• Stay updated on emerging AI testing methodologies, tools, and best practices, contributing proactively to process improvements within the testing team.  

Qualifications & Skills:

• Proven experience in software testing, preferably with a focus on AI application testing.  
• Familiarity with Gen AI concepts including Large Language Models (LLMs), prompt engineering, and hallucinations.  
• Strong understanding of testing methodologies, tools, and techniques with experience in test automation.  
• Familiarity with programming languages such as Python, Java, or JavaScript.  
• Prior experience in API Automation Testing is highly desirable.  
• Experience with testing frameworks and tools such as Selenium, Appium, Cypress, or similar.  
• Excellent analytical and problem-solving skills with strong attention to detail.  
• Strong communication and collaboration skills; ability to work effectively in a team environment.  
• Proven ability to multitask and manage priorities in a fast-paced, dynamic environment.  
• Knowledge of machine learning, Open APIs, LangChain concepts, and algorithms is a plus.