Data Annotator

We are seeking Function Calling Annotators to support a large-scale annotation project. The role involves evaluating user queries, function signatures, and function calls to ensure correct and accurate mapping of intent to execution. Annotators will not write code but must be able to understand programmatic logic, parameters, and structured data (JSON/XML).

This is a role with certification-based onboarding.

Key Responsibilities

• Review user queries in English (US locale) and identify user intent.

• Examine corresponding function signatures (parameters, input/output).

• Evaluate given function calls against the query intent for:

o Correctness of parameters

o Groundedness (values taken from query/context)

o Completeness (all required inputs covered)

o Relevance (appropriate function chosen)

• Annotate audit questions with clear judgments (Pass/Fail, Error Type).

• Flag issues such as ungrounded assumptions, missing parameters, or incorrect function selection.

• Maintain high accuracy and throughput on assigned batches.

• Collaborate with QA/Team Lead for feedback loops and continuous improvement.