Outcome.
Did the end state match the objective? Necessary, insufficient, and the only thing most agent benchmarks report.
MKC2 results ↗An agent that reaches a correct result by reading records it should not have touched has failed. Agent evaluation scores the whole trajectory.
01 / DEFINITION
The short answer first.AI agent evaluation measures a multi-step system by its whole trajectory: the actions taken, their order, the data touched and the final state produced. Unlike single-response evaluation, it has to account for side effects, because an agent changes the world as it works.
Three properties make agent evaluation harder than model evaluation. The action space is combinatorial, so exhaustive coverage is impossible. Side effects are real, so evaluation needs a sandbox with realistic state rather than a static test set. And success is multidimensional: the same end state can be reached efficiently, wastefully, or through a path that breached a boundary.
WORKED EXAMPLE
A scheduling agent says it moved a service appointment from Thursday to Friday. The text is correct only if the booking system contains the new appointment, the old slot has been released and the customer received the intended confirmation. An evaluation that reads only the final chat message can miss a duplicate booking or a change applied to the wrong customer.
Create a test account with an existing Thursday reservation. Give the agent permission to change that reservation and no permission to modify another account. Run the task against a disposable scheduling service and record the tool calls, arguments, results and final bookings. Grade the resulting state independently of the agent's explanation. A polished confirmation cannot compensate for an appointment that remains on Thursday.
Now make the booking service accept the update but delay its response. The agent may retry after a timeout. Check that the retry recognises the original operation and does not reserve Friday twice. Repeat with a genuine rejection, where the expected result is an unchanged booking and a useful explanation rather than a fabricated confirmation.
Include a conflicting request that names another customer's booking. The appropriate outcome is a safe refusal or escalation, even if the system could technically reach the record. Keep task completion, correct refusal, side effects and recovery as separate results. These distinctions tell the release owner whether to improve planning, tool design, permissions or the wording presented to the customer.
02 / WHAT TO SCORE
Did the end state match the objective? Necessary, insufficient, and the only thing most agent benchmarks report.
Was every action permitted and appropriate? An agent that read records outside its scope failed, whatever the outcome.
Steps, tokens and tool calls against a reasonable baseline. Waste correlates with confusion and predicts failure on harder instances.
What happened after a tool error or an empty result? Retrying identically until a step limit is the most common failure pattern in practice.
Did the agent stop when it could not proceed safely, or did it improvise? Correct refusal is a capability and should be scored as one.
Path validity is where evaluation meets security. An agent that produces correct answers while routinely reading beyond its scope will eventually be given a task where that behaviour is harmful. Scoring the path makes the tendency visible while it is still cheap, which is why agent evaluation belongs alongside AI agent permissions rather than in a separate quality workstream.
03 / BUILDING THE HARNESS
| Component | Purpose | Common shortcut and its cost |
|---|---|---|
| Sandboxed state | Realistic data the agent can safely modify | Testing against production; unrepeatable and risky |
| Deterministic tools | Same call, same response, every run | Live APIs; failures cannot be attributed to the agent |
| Injected faults | Timeouts, empty results, malformed responses | Happy-path only; recovery behaviour never measured |
| Adversarial content | Documents carrying hidden instructions | Clean corpora; injection resistance untested |
| Trajectory log | Every step, input and decision, replayable | Final-answer logging; no way to diagnose a failure |
Fault injection is the component with the best return and the lowest adoption. Most agent failures observed in production are not reasoning failures but handling failures: a tool returned an error, the agent treated the error text as data, and the trajectory degraded from there. A harness that never returns an error has not tested the majority of real failure modes.
04 / FROM EVALUATION TO AUTONOMY
Agent evaluation produces a distribution of behaviour, not a guarantee. The useful output is not a pass or fail but a mapping from measured evidence to permitted authority, per capability: this action type has a strong record and may proceed with logging; that one is proposal-only until a sustained record exists.
Privilege AI applies the same structure to model supervision, where authority is set per check rather than per model — 25 of MKC2's 45 checks carry report-only authority and one is refused outright. The agent case is the same argument applied to actions instead of findings, and it is developed in AI agent governance and AI model supervision.
A supervision model can also review agent actions directly, which is part of what MKC2 is built to do: evaluate AI-generated work and the actions of AI agents, in review-only mode.
05 / QUESTIONS
The unit of analysis. Model evaluation scores a response; agent evaluation scores a sequence of actions with side effects. An agent can produce a correct answer through an impermissible path, and only trajectory-level scoring will show it.
Coverage should be organised by failure mode rather than by count. A few dozen scenarios spanning normal operation, tool failure, ambiguous instructions, adversarial content and out-of-scope requests is more informative than hundreds of happy-path variations.
Partially, through logged trajectories and sampled human review of proposed actions. What production cannot provide is a controlled comparison, because state differs on every run. Both are needed: the sandbox for attribution, production for realism. See AI agent monitoring.
Yes. Step count, token use and wall-clock time are quality signals as well as budget signals: a run that takes four times the baseline steps is usually confused, and the excess predicts failure on harder instances.
The domain expert who owns the process, working with whoever owns access control. The first defines what should happen; the second defines what is permitted. Writing both down is most of the evaluation design.
WORK WITH PRIVILEGE AI
Know how your agent reaches its answer.
We assess the steps an agent takes, not just whether the final answer looks right.
Trust based on evidence.
We measure what an agent does well and where it falls short, so you can decide how much to let it do.
A second check on AI work.
We build supervision that reviews AI-generated work and agent actions before they reach your team.
Talk with Privilege AI
Describe a task your agent must finish, the tools it may use and the final state your team considers correct.
CONTACT PRIVILEGE AI
Describe a task your agent must finish, the tools it may use and the final state your team considers correct.
Privilege AI evaluates AI-generated work and the actions of AI agents, with authority set by measured evidence.