Senior QA Specialist - Business Analytics

Job Description

Configure and test AI/ML product/system. Perform manual testing to identify and document quality issues. Perform exploratory and black box testing for AI features. Assist in testing AI-specific scenarios such as prompt stability and hallucination edge cases. Change system configurations to extract the best quality output. Manage and review the deliveries of a team of external team of manual QA. Collaborate with developers, product owners, and other QA engineers in Agile teams. Design, develop, and execute test cases for functional and non-functional requirements. Perform API, integration, acceptance, system, and end-to-end (E2E) testing. Write and maintain basic level of automation test scripts using tools like Selenium, Playwright, or similar, preferably in Python. Conduct regression, sanity, and smoke testing during CI/CD deployments. Ensure adequate test coverage for backend services and microservices. Must-Have Skills Attention to details Aptitude to learn and understand requirements even when clear documentation may not be available Experience with Testing Methods: o Black-box, White-box, and Grey-box testing Scripting knowledge in Python Strong experience in Functional Testing: o API Testing (Postman, REST Assured, etc) o Integration Testing o Acceptance Testing o End-to-End Testing o Smoke and Sanity Testing o System Testing Good understanding of Non-Functional Testing: o Load and Performance Testing (eg, JMeter, Locust) o Stress Testing o Security Testing basics Regression testing strategies and best practices Good-to-Have (AI Testing Focus) Familiarity on testing AI/ML products/systems. Familiarity with CI/CD pipelines (Azure DevOps, Gitlab, GitHub Actions, etc) Understanding of hallucination and prompt variability in LLMs. Experience or interest in RAG (Retrieval-Augmented Generation) system testing. Familiarity with evaluating LLM (large language model) outputs. Familiarity with data labeling and validation practices. Qualifications bachelors degree in Computer Science, Engineering, or related field. ISTQB certification or equivalent is a plus.