Senior Test Automation Engineer

Senior Automation Test Engineer

Key responsibilities

• Layout the end-to-end test strategy: unit, API, UI, contract, smoke, and release gates.
• Implement durable automation (Playwright or equivalent + API frameworks) for critical user journeys.
• Design AI response testing: golden sets, prompt/playbook regressions, structured-output checks, grounding/citation asserts.
• Build eval harnesses for LLM / AI workflow outputs - scoring, thresholds, failure triage, and regression gates.
• Cover retrieval-quality checks where the product needs them, without owning the data platform.
• Wire suites into CI/CD with flaky-test discipline and clear dashboards for engineering and release owners.
• Mentor the team on risk-based coverage and maintainable fixtures for tenant or multi-env data.

Must-have experience

• B.tech Degree
• 10+ years QA / SDET; owned whole-product test strategy at senior level.
• Strong coding (TypeScript, C#, or Python) for automation and eval harnesses.
• Proven LLM / AI output evaluation (golden datasets, rubrics, regression gates).
• CI/CD (Azure DevOps / GitHub Actions) and API + UI automation at scale.
• Track record turning flaky suites into trusted release signal.

Nice to have

• Experience testing RAG or hybrid search quality.
• Security smoke (OWASP) and performance smoke in the same pipeline.
• Ephemeral, seeded test environments.