Job Title AI Test Manager
Job Summary
We are seeking a skilled AI Test Manager who will be responsible in leading a team of QE professionals in designing, implementing, and executing testing strategies for AI-based applications and systems. This role requires a strong understanding of AI/ML concepts, testing methodologies, and project management skills to ensure the delivery of high-quality products. The AI Test Manager will collaborate with cross-functional teams, including data scientists, software engineers, and product managers, to ensure robust testing processes and practices are in place.
Key
-
Leadership and Team Management
-
- Lead and mentor a team of QA engineers focused on AI and machine learning projects.
- Foster a collaborative and innovative team culture.
- Conduct performance evaluations, provide feedback, and support career development for team members.
-
Testing Strategy Development
-
- Design and implement comprehensive testing strategies for AI applications, including functional, integration, regression, and performance testing.
- Develop and maintain test plans, test cases, and test scripts tailored to AI-specific challenges.
- Ensure compliance with industry standards and best practices in AI testing.
-
Collaboration with Cross-Functional Teams
-
- Work closely with data scientists, developers, and product managers to understand requirements and provide insights on testing needs.
- Participate in design reviews and provide input on potential testing challenges and considerations.
-
AI Model Validation
-
- Define and execute methodologies for validating AI models, including bias detection, performance evaluation, and accuracy testing.
- Establish metrics and KPIs to assess the effectiveness of AI systems and ensure they meet specified requirements.
- Experience in testing LLM models
-
Test Automation and Tools
-
- Oversee the implementation and maintenance of automated testing frameworks and tools specific to AI applications.
- Stay updated on emerging tools and technologies in AI testing and recommend their adoption as appropriate. Knowledge on explainability testing tools (e.g., LIME, SHAP etc.)
- Experience in any of the AI testing platforms and tools (e.g., Functionize, Applitools, Testim etc.)
- Knowledge on cloud platforms & services for ML Ops / LLM Ops
-
Documentation and Reporting
-
- Maintain detailed documentation of testing processes, results, and lessons learned.
- Prepare and present reports to stakeholders on testing progress, risks, and outcomes.
-
Continuous Improvement
-
- Identify opportunities for process improvement in testing and development methodologies.
- Advocate for best practices in AI testing across the organization.
-
Risk Management
-
- Identify potential risks in AI models and applications and develop mitigation strategies.
- Ensure timely communication of testing issues and risks to project stakeholders.
Qualifications
-
Education
Bachelor s degree in Computer Science, Engineering, or a related field; Master s degree preferred.
Experience
- 10+ years of experience in software testing / development with exposure to Python
- 1-2 years of experience in defining test strategy and plan for AI/ML applications
- 2+ years in a managerial or leadership role.
Technical Skills
- Strong understanding of AI/ML concepts, algorithms, and frameworks (e.g., TensorFlow, PyTorch).
- Knowledge on ML / LLM frameworks & libraries
- Proficiency in programming languages commonly used in AI testing (e.g., Python, R).
- Experience with test automation tools (e.g., Selenium, TestNG) and frameworks.
- Familiarity with data validation, model evaluation, and statistical analysis.
Soft Skills
- Excellent communication and interpersonal skills.
- Strong problem-solving abilities and analytical thinking.
- Ability to work collaboratively in a fast-paced, dynamic environment.
