QA Lead – AI / ML Systems

About the Role

Key Responsibilities

• Own and drive QA strategy for AI systems across the model, data, and product lifecycle.
• Lead AI model validation efforts, covering LLM/RAG testing, bias analysis, and performance evaluation.
• Define and implement AI Evaluation Metrics (accuracy, fairness, drift, explainability) aligned with business and regulatory expectations.
• Establish frameworks for Explainability Testing (SHAP, LIME, XAI) and ensure interpretability of AI outcomes.
• Collaborate with Data Science and MLOps teams to validate models within cloud environments (Azure ML, AWS Sagemaker, Vertex AI).
• Drive verification and validation (V&V) for AI models and applications under FDA and SaMD compliance frameworks.
• Ensure test traceability, documentation, and audit readiness in line with ISO 13485, IEC 62304, and ISO 14971.
• Develop Python-based automation for AI testing, data validation, and model evaluation pipelines.
• Provide technical leadership to QA teams, ensuring alignment with AI/ML development best practices.
• Collaborate with cross-functional teams (Product, Data, Regulatory, Engineering) to identify risks, gaps, and opportunities for test optimization.
• Present validation findings, risks, and recommendations to senior stakeholders and regulatory reviewers.

Required Qualifications

• Bachelors or Masters degree in Computer Science, Data Engineering, Biomedical Engineering, or related field.
• 8-12 years of total QA experience, with 23 years directly in AI/ML or GenAI QA.
• Proven experience in AI Model Validation, LLM/RAG Testing, and AI Evaluation Metrics.
• Strong knowledge of MLOps concepts and cloud platforms (Azure ML, AWS Sagemaker, Vertex AI).
• Understanding of FDA AI/ML Compliance, SaMD testing, and regulated software QA.
• Hands-on expertise in Python automation and testing AI/ML pipelines.
• Excellent documentation and communication skills with ability to produce traceable validation artifacts.