AI Agent Evaluation Analysts

Senior AI Agent Evaluation Analyst — 7+ Years Experience Required

About the Role:

experienced AI Agent Evaluation Analysts

This role is central to improving how next-generation AI systems reason, follow policies, and interact with real-world data.

Responsibilities:

• Evaluate AI agent decision paths (“trajectories”) against golden standards and policies.
• Detect logic gaps, unrealistic assumptions, and policy violations in AI reasoning.
• Analyze structured data (JSON/YAML logs) to verify correct tool usage and decision flows.
• Annotate and categorize reasoning errors, causal chains, and interaction quality.
• Suggest improvements to test designs and policy coverage.

Requirements:

7+ years

AI task annotation

✅ Strong command of analytical reasoning and documentation.

✅ Familiarity with structured datasets and annotation platforms.

✅ Exceptional written English and logical clarity.

Nice to Have:

• Experience with OpenAI, Anthropic, or LangChain-based systems.
• Prior involvement in multi-agent or tool-based reasoning evaluations.
• Deep understanding of policy adherence, hallucination control, or task consistency.

Location:

E: sajid.ahmed@truelancer.com

Think you’ve got the brain for it?

AI Agent Evaluation Analyst Application Form