QA Engineer (Product)

QA Engineer

Key Responsibilities

• Collaborate with product managers, AI engineers, and developers to define QA strategies for LLM and voice-based features.
• Design and execute test cases for functional, regression, integration, and performance testing.
• Test conversational AI/LLM features for accuracy, response relevance, context retention, hallucination detection, and prompt robustness.
• Conduct voice testing across multiple platforms, focusing on speech recognition, TTS, latency, and conversational flow validation.
• Build and maintain automated regression test suites (preferred stack: Selenium + Java/TestNG, with exposure to Cypress/Playwright).
• Perform API testing using Postman/RestAssured, including validation of AI/voice APIs.
• Define and track QA metrics such as response accuracy, latency benchmarks, and defect leakage.
• Collaborate with cross-functional teams to continuously improve test coverage and product reliability.

Required Skills & Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field.
• 3–5 years of proven QA experience (manual + automation).
• Strong knowledge of QA methodologies, SDLC, STLC, and Agile/Scrum practices.
• Hands-on experience with automation frameworks (Selenium, Cypress, or Playwright).
• Experience testing LLM-based features (OpenAI, Anthropic, or similar APIs).
• Practical knowledge of voice testing, including ASR (Automatic Speech Recognition), NLP pipelines, and TTS validation.
• Strong analytical and communication skills with the ability to think from the end-user perspective.

Good to Have

• Experience with performance testing (JMeter, Locust) for LLM and voice workloads.
• Familiarity with evaluation metrics for AI systems (e.g., BLEU, ROUGE, WER for speech).
• Hands-on with cloud-based AI testing tools and platforms (BrowserStack, Sauce Labs, or Speech-to-Text benchmarks).
• ISTQB or other QA certifications.

About Gnani.ai

Gnani.ai is a Gen AI-based startup that is working on voice and chat based solutions for large businesses. Gen AI is the biggest innovation that is disrupting the market and we are at the heart of this disruption. Funded by one of the largest global conglomerates in the world, and backed a number of market leaders in the tech industry.

We work with top companies across banking, insurance, e-commerce, and financial services, delivering scalable AI solutions that elevate customer experience and operational efficiency. With strong growth momentum, Gnani.ai is rapidly expanding its global footprint.

India AI Mission

If you’re passionate about GenAI and want to build cutting-edge technologies in a fast-paced, high-growth environment, join us. We’re looking for driven individuals ready to take on big challenges and shape the future of AI.