Functional Test Engineer – Agentic AI Solution (Banking & Wealth Management)

Job Title: Functional Tester – Agentic AI Solution (Banking & Wealth Management)

Location: Noida

Experience Level: 2–6 years

Reports To: QA Lead / Product Manager

We are looking for an experienced Functional Tester to validate the business workflows, compliance logic, and customer-facing features of an Agentic AI solution built for the banking and wealth management domain. This solution leverages LLMs, retrieval-augmented generation (RAG), and domain ontologies to provide intelligent data retrieval, client interaction, and regulatory compliance automation.

Key Responsibilities

Functional Testing

• Understand functional and non-functional requirements for Agentic AI features (e.g., document Q&A, investment summarization, client onboarding, compliance tracking)
• Validate end-to-end user journeys across banking and wealth advisory use cases
• Test conversational workflows and agentic logic flows (multi-step LLM tasks)
• Verify outputs from LLMs for relevance, consistency, and context-awareness
• Ensure correct role-based access across advisors, relationship managers, and compliance officers

Test Planning & Execution

• Prepare detailed test plans, test cases, and traceability matrices from EPICs, features, and user stories
• Execute manual testing for user interfaces, APIs, and chat/agent interactions
• Perform regression testing across AI pipelines after prompt or model updates
• Log, track, and retest defects using tools like Azure DevOps, Jira, or TestRail
• AI Model & RAG Testing
• Validate grounding of LLM responses from vector databases (e.g., Azure AI Search, Pinecone)
• Check performance of RAG pipelines including hallucination mitigation, context window boundaries, and fallback logic
• Test interaction of agent orchestration flows (LangChain/Autogen/semantic kernel)
• Verify prompt-based test cases for accuracy and coverage
• Integration & Data Testing
• Validate PDF/statement extraction logic from custodians (e.g., Morgan Stanley, Pershing)
• Check structured data output (JSON, tables) from Azure Form Recognizer or OpenAI pipelines

Ensure data integrity and lineage from ingestion → processing → user response

Must-Have Skills & Tools Category Tools/Tech Test Management Azure / AWA DevOps, Jira, TestRail, Azure Board Test Types Functional, UI, Integration, AI behavior Domain Knowledge Wealth Mgmt workflows, KYC, portfolio views, regulatory rules AI Concepts LLM output validation, RAG testing, prompt engineering basics Data JSON validation, SQL for test data setup Tools (Optional) Postman, Python (for test automation scripts), Excel-based test mapping

Preferred Qualifications

• Bachelor’s degree in Engineering, Finance, or Computer Science
• Experience testing AI/ML solutions or conversational AI products
• Familiarity with Azure AI stack, OpenAI, or AWS Bedrock
• Exposure to compliance-heavy domains like banking, wealth, or insurance
• Ability to work with cross-functional Agile teams (product, engineering, AI/ML, compliance)

Nice-to-Have Skills

• Experience writing prompt-based test cases for LLMs
• Familiarity with agentic workflows (e.g., task chaining, autonomy testing)
• Understanding of financial statement formats (portfolio summaries, trade confirms)
• Basic scripting to validate data output (e.g., Python, bash)