QA Lead AI/ML Systems

As a QA Lead AI Systems at Codvo, your role involves leading the validation and quality assurance of AI-powered products, including LLM/RAG systems, Software as a Medical Device (SaMD), and other AI-driven digital solutions. You will be responsible for establishing end-to-end QA frameworks, validation strategies, and compliance processes for AI/ML models across development and deployment pipelines. Your expertise in bridging data science, software engineering, and regulatory QA is crucial to ensure that AI systems meet performance, safety, and compliance standards. **Key Responsibilities:** - Own and drive QA strategy for AI systems across the model, data, and product lifecycle. - Lead AI model validation efforts, covering LLM/RAG testing, bias analysis, and performance evaluation. - Define and implement AI Evaluation Metrics (accuracy, fairness, drift, explainability) aligned with business and regulatory expectations. - Establish frameworks for Explainability Testing (SHAP, LIME, XAI) and ensure interpretability of AI outcomes. - Collaborate with Data Science and MLOps teams to validate models within cloud environments (Azure ML, AWS Sagemaker, Vertex AI). - Drive verification and validation (V&V) for AI models and applications under FDA and SaMD compliance frameworks. - Ensure test traceability, documentation, and audit readiness in line with ISO 13485, IEC 62304, and ISO 14971. - Develop Python-based automation for AI testing, data validation, and model evaluation pipelines. - Provide technical leadership to QA teams, ensuring alignment with AI/ML development best practices. - Collaborate with cross-functional teams (Product, Data, Regulatory, Engineering) to identify risks, gaps, and opportunities for test optimization. - Present validation findings, risks, and recommendations to senior stakeholders and regulatory reviewers. **Required Qualifications:** - Bachelors or Masters degree in Computer Science, Data Engineering, Biomedical Engineering, or related field. - 8-12 years of total QA experience, with 2-3 years directly in AI/ML or GenAI QA. - Proven experience in AI Model Validation, LLM/RAG Testing, and AI Evaluation Metrics. - Strong knowledge of MLOps concepts and cloud platforms (Azure ML, AWS Sagemaker, Vertex AI). - Understanding of FDA AI/ML Compliance, SaMD testing, and regulated software QA. - Hands-on expertise in Python automation and testing AI/ML pipelines. - Excellent documentation and communication skills with the ability to produce traceable validation artifacts.,