AI QA Test Engineer

3 - 5 years

6 - 10 Lacs

Posted:11 hours ago| Platform: Naukri logo

Apply

Work Mode

Work from Office

Job Type

Full Time

Job Description

About the Role

AI QA Test Engineer

Participates in QA activities on project assignments and ensures timely delivery on QA commitments in a fast-paced product development environment.

Review of functional requirements and technical design documentation and creation of test plans, test scenarios and test cases. Execution of test plans and certification of application releases

Key Responsibilities:

AI/ML Testing

  • Validate ML models, data pipelines, and inference APIs for accuracy, fairness, and robustness
  • Evaluate model behavior across diverse data sets, edge cases, and unexpected inputs
  • Monitor model drift, performance degradation, and bias over time
  • Conduct prompt testing, hallucination detection, and safety validation for LLM-based systems
  • Experience of testing implementation of AI/ML into application
  • Manual Testing with automation exposure
  • API testing with SOAP UI/ Postman
  • Agile methodology experience
  • Test & Defect management with JIRA/ Azure DevOps (ADO)
  • DB Testing Experience  - Good to Have                                                                              
  • Tools - Deepeval/ Langfuse/ Pytest

Quality Engineering & Automation

  • Build and maintain automated test suites for AI features
  • Develop test scripts for ML workflows using Python, Playwright, and CI/CD tools
  • Design and execute manual and automated tests for functional, performance, and security scenarios
  • Identify defects, establish test metrics, reporting dashboards, and test traceability

Data Quality & Responsible AI

  • Validate training and inference data quality, labeling consistency, and data governance
  • Work with data science and product teams to implement responsible AI testing guidelines
  • Document and escalate risks related to bias, privacy, explainability, and compliance

Cross-Functional Collaboration

  • Partner with Data Scientists, ML Engineers, and Product Managers to define acceptance criteria
  • Participate in design reviews, model evaluations, and release readiness checkpoints
  • Contribute to QA strategy and Test Plans

Required Qualifications

  • Bachelor’s degree in computer science, Engineering, or related field (or equivalent experience)
  • 4+ years of experience in software QA engineering
  • 2+ years working with AI/ML or data-centric systems
  • Strong Python programming skills
  • Proficiency in automation frameworks (e.g., Playwright, WebdriverIO, Selenium)
  • Experience with ML platforms (TensorFlow, PyTorch, Hugging Face, MLflow, etc.)
  • Hands-on experience testing APIs, data pipelines, and distributed systems
  • Exposure to RAG pipelines and agent frameworks
  • Familiarity with prompt testing and LLM evaluation techniques
  • Experience with model evaluation metrics (precision, recall, F1, ROC-AUC, BLEU, perplexity, etc.)
  • Knowledge of MLOps, CI/CD, and cloud platforms (AWS, Azure, or GCP)
  • Familiarity with Responsible AI frameworks (NIST AI RMF, ISO/IEC standards, fairness toolkits)
  • Experience testing vector databases and AI search (Pinecone, Elasticsearch, Redis Vector)

Key Skills

  • AI/ML testing & analysis
  • Automation & scripting
  • Data validation & pipelines
  • Risk, ethics, & compliance mindset
  • Strong problem-solving & debugging skills
  • Excellent documentation & communication skills
  • Working with distributed teams

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
ORION SYSTEMS logo
ORION SYSTEMS

Information Technology

Tech City

RecommendedJobs for You

coimbatore, tamil nadu, india