QA Automation Engineer

We are seeking a highly skilled QA Tester with 78 years of experience in both manual and automated testing, to join our GenAI product team. The ideal candidate will have strong expertise in Python Selenium using BDD frameworks, be comfortable designing and executing prompt engineering test cases, and have hands-on experience with Azure DevOps (ADO) Boards for test case management.

Responsibilities

- Design, develop, and execute manual test cases and build automated scripts using Python Selenium with BDD frameworks (e.g., Behave, Cucumber).

- Collaborate closely with product managers, developers, and AI engineers to define acceptance criteria and testing strategies for GenAI features.

- Conduct prompt engineering test validation to assess model outputs, relevance, and accuracy under varying input conditions.

- Set up, configure, and maintain Azure DevOps Boards for managing requirements, test cases, defects, and release cycles.

- Perform regression, functional, integration, and user acceptance testing for multiple releases in an agile environment.

- Contribute to continuous improvement of testing practices, automation coverage, and reporting dashboards.

- Troubleshoot test failures, identify root causes, and document detailed defect reports.

- Ensure product quality across web interfaces, AI model responses, and API integrations.

- Collaborate with cross-functional teams in sprint planning, daily stand-ups, and retrospectives.

Required Skills & Experience

- 7–8 years of overall QA experience (manual + automation).

- Strong proficiency in Python and Selenium WebDriver with BDD frameworks like Behave or Cucumber.

- Solid experience designing prompt engineering test cases for AI/ML model evaluation.

- Hands-on experience with Azure DevOps (ADO) Boards for end-to-end test case tracking.

- Excellent understanding of software QA methodologies, tools, and processes.

- Proven ability to work in Agile/Scrum frameworks with continuous integration and delivery pipelines.

- Ability to analyze requirements and translate them into clear, testable scenarios.

- Strong debugging and problem-solving skills.

Preferred Qualifications

- Experience in testing GenAI or LLM-based applications.

- Knowledge of REST API testing tools and AI evaluation metrics.

- Familiarity with cloud platforms (Azure, AWS, GCP) for testing distributed systems.