Role & responsibilities Key Responsibilities: AI Testing Strategy and Planning Collaborate with cross-functional teams to develop comprehensive AI testing strategies and plans for AI-powered applications. Work closely with product managers, data scientists, and developers to understand AI model requirements, use cases, and project goals. Define the scope and objectives of AI testing efforts, including performance, accuracy, bias detection, and robustness of AI models. Test Execution for AI Models and Algorithms Design, develop, and execute test cases for AI systems and models (including machine learning and deep learning algorithms). Test and validate AI solutions across various stages of the development lifecycle, including model training, testing, and deployment. Ensure that AI models meet business requirements and perform accurately under various real-world conditions. Evaluate the performance of AI models by assessing speed, efficiency, scalability, and resource utilization. Perform manual and automated testing on AI-based applications, platforms, and solutions. AI Model Accuracy and Validation Test AI models for accuracy, precision, recall, F1 score, and other performance metrics. Ensure AI models' fairness by conducting tests for potential bias in decisionmaking processes, especially in clinical or medical applications. Validate AI model predictions against real-world data, ensuring that results are consistent, reliable, and actionable. Also, need to look the test results from a business perspective and help evaluate the balance between risks and benefits. Collaboration and Knowledge Sharing Work with data scientists, AI engineers and Test Manager to improve testing methodologies and continuously optimize AI model testing processes. Provide feedback on AI models, pointing out any potential improvements in testing coverage or areas for model retraining. Communicate findings, bugs, and issues related to AI models to technical teams, ensuring prompt resolution. Help the team set up AI Testing standards, make informed decisions, and build knowledge across projects Help the team in decision-making processes, such as whether to continue or stop investments based on testing results. Test Automation for AI Projects Develop and implement automated testing scripts and frameworks specifically designed for AI applications. Utilize AI testing tools and frameworks (RAGAS etc.) to automate the validation of AI models and algorithms. Integrate automated AI testing within continuous integration and continuous deployment (CI/CD) pipelines. Compliance and Regulatory Testing Ensure that AI applications comply with industry-specific regulations, especially in the pharma and healthcare sectors (e.g., FDA regulations, HIPAA compliance). Verify that all AI-driven processes adhere to ethical standards and data privacy laws. Continuous Improvement and Research Stay up-to-date with the latest trends, tools, and techniques in AI testing and apply these advancements to optimize the testing process. Participate in AI testing forums and workshops, contributing insights to improve best practices within the team. Reporting and Documentation Document test results, methodologies, and issues clearly, providing insights into test coverage, risk analysis, and performance benchmarks. Prepare detailed reports for both technical and non-technical stakeholders, summarizing testing outcomes and potential risks associated with AI implementations. Assist in the creation and maintenance of knowledge-sharing platforms related to AI testing best practices. Key Skills and Qualifications: Technical Expertise Strong knowledge of AI/ML testing methodologies and best practices. Experience with any AI development frameworks and libraries such as TensorFlow, Keras, PyTorch, scikit-learn, RAGAS and MLlib. Experience in testing tools and environments for AI-based systems (e.g., Jupyter Notebooks, Apache Spark, and DataRobot ). Experience with performance testing tools like Grafana K6 and JMeter for AI solutions. Knowledge of Python (Must to have), R, JavaScript or other programming languages frequently used in AI/ML. Knowledge of cloud technologies like Microsoft Azure / AWS. Understanding of test automation frameworks and experience in tools like Cypress , Playwright and Pytest for automating AI tests. AI Model Evaluation Solid understanding of machine learning and deep learning models, including supervised and unsupervised learning techniques. Familiarity with evaluating AI models on metrics such as accuracy, precision, recall, F1 score, confusion matrices, and AUC. Ability to identify and test for model biases, fairness, and ethical implications, especially in sensitive applications like healthcare and pharma. Analytical and Problem-Solving Skills Strong problem-solving abilities and keen attention to detail, with a systematic approach to diagnosing and resolving AI-related issues. Ability to perform root cause analysis of issues in AI algorithms and suggest actionable fixes. Collaboration and Communication Excellent teamwork and communication skills, with the ability to collaborate with cross-functional teams, including data scientists, engineers, and product managers. Strong verbal and written communication skills to convey technical information clearly and concisely to both technical and non-technical stakeholders. Experience Minimum of 8+ years experience in software testing, with at least 2 years focused on testing AI/ML models or AI-based applications. Proven experience in testing AI/ML algorithms in production or staging environments. Experience in testing Visual AI Assistant Applications is good to have. Experience working in a regulated industry (such as pharmaceuticals or healthcare) is a plus. Preferred Qualifications: Experience with cloud platforms (e.g., AWS, Azure) for deploying AI applications and models. Certification in AWS/Azure will be good to have. Familiarity with DevOps practices and integrating AI testing into CI/CD pipelines. Certification in AI/ML or related testing frameworks (e.g. ISTQB AI Tester) This AI Tester role is a unique opportunity to shape the future of AI in the pharmaceutical industry. If youre passionate about AI, testing, and making a difference in healthcare, we encourage you to apply. Preferred candidate profile
