Sample trajectories on a schedule.
A fixed number of complete runs read end to end each week. Dashboards show aggregates; reading a whole trajectory is what surfaces behaviour nobody thought to measure.
MKC2 results ↗A record of what an agent called is not enough. Monitoring has to capture what it was reacting to, or no incident can ever be explained.
01 / DEFINITION
The short answer first.AI agent monitoring is the continuous recording and inspection of what an agent does: every tool call, the content that prompted it, the data returned, and the resulting action. Its purpose is not uptime but explanation — being able to reconstruct why a particular action was taken.
Ordinary application monitoring answers whether a system is working. Agent monitoring has to answer a harder question after the fact: why did this system do that? Answering it requires the inputs that shaped the decision, not just the calls that resulted, because the cause of an unwanted action is frequently a piece of retrieved content rather than a bug.
WORKED EXAMPLE
A customer service agent proposes a refund and the payment service accepts it. A monitor that records only a successful tool response cannot establish whether the correct order was refunded, whether the amount matched approval or whether a second retry created another payment. Trace the individual operation through proposal, authorisation, execution and settlement.
Use a correlation identifier that links the agent run, policy decision and payment service receipt. Store the minimum information operators need to investigate. A full card number or an unrestricted transcript does not belong in an ordinary monitoring dashboard. Count completed business transactions alongside attempted tool calls. Two calls can represent one retry-safe refund, and one apparently successful call can still correspond to an unsettled payment.
Test a delayed settlement and a rejected refund. The operator should see distinguishable states and a clear owner for each exception. If the system reports every delay as a failure, it may encourage unnecessary retries. If it treats acceptance as final settlement, the customer may receive a confirmation before the funds have actually moved.
At review time, sample apparently successful refunds as well as alerts. Compare the authorised amount with the final receipt and check that the agent's explanation matches the payment record. This reveals confidently completed mistakes that an exception-only dashboard cannot show.
02 / WHAT TO RECORD
A useful trajectory record is replayable. For each step it holds the objective, the identity the agent is acting as, the tool called with its arguments, the response received, the model's stated reasoning where available, and a reference to the retrieved content in play. Content references matter: when an injected instruction is the cause, the evidence is in the document, and a log without a pointer to it cannot show what happened.
Two practical constraints apply. Full content logging conflicts with data minimisation, so the usual compromise is content hashes and identifiers with the source retrievable from the system of record. And trajectory logs contain the same sensitive material as the agent's inputs, so they inherit its access controls — a log store readable by the whole engineering team can undo the retrieval boundaries the agent respects.
| Signal | What a change indicates |
|---|---|
| Steps per task | Confusion, looping, or a task distribution shift |
| Tool error rate | A dependency changed; recovery behaviour is now in play |
| Scope-boundary events | Attempts to read or act outside permission, successful or not |
| Repeated identical calls | A retry loop the agent cannot exit |
| Proposal rejection rate | Human reviewers disagreeing more often; the leading quality indicator |
| Egress attempts | Outbound requests to destinations not on the allowlist |
03 / SCOPE EVENTS ARE THE IMPORTANT ONES
Access-control denials for an agent are not routine noise. They record a system attempting something it was not meant to attempt.
In conventional applications a permission denial is usually a misconfiguration or a user mistake. For an agent it means the model decided to do something outside its scope, and the reason matters: an ambiguous instruction, a confusing tool description, a genuine capability gap, or injected content steering it. Each has a different fix, and all of them are cheaper to find at the denial than at the incident.
Treating denials as findings rather than as noise requires them to be logged with full context and reviewed on a cadence. The rate is also a governance signal: an agent whose scope-boundary events are rising is drifting away from the task its permissions were designed around, which is usually a prompt to revisit AI agent permissions.
04 / REVIEW AND RETENTION
A fixed number of complete runs read end to end each week. Dashboards show aggregates; reading a whole trajectory is what surfaces behaviour nobody thought to measure.
Boundary events and egress attempts should page someone. Quality signals such as rejection rate belong in a weekly review, not in an alert.
Incidents surface weeks later. Retention has to outlast discovery time, while respecting the retention rules of the underlying data.
A supervision model can also read trajectories, which is part of what MKC2 does: it evaluates AI-generated work and the actions of AI agents, in review-only mode. That makes sampled review affordable at volumes where a human sample would be too thin to be informative — provided the supervisor's own ranking quality has been measured, which is the argument in AI model supervision.
05 / QUESTIONS
Model monitoring watches output quality and input distribution. Agent monitoring watches actions, access and trajectories. An agent can produce perfectly good text while reading records it should not touch, and only the second kind of monitoring shows it. See AI model monitoring.
Structure in full; content by reference where the content is sensitive. Hashes and document identifiers preserve the ability to reconstruct a run without duplicating confidential material into a second store with different access controls.
Long enough that an incident discovered weeks later can still be investigated, and no longer than the retention policy of the underlying data permits. Those two constraints often conflict, and resolving it deliberately is better than defaulting to whatever the logging tool does.
It can detect the effects: unexpected tool calls, scope-boundary events, egress attempts, sudden changes in behaviour mid-run. Detecting the injected instruction itself requires inspecting retrieved content, which is why trajectories need to reference their sources. See AI agent prompt injection.
Whoever owns the agent, on a schedule, plus security for boundary and egress events. Logs nobody reads are storage cost, not a control, and the weekly sampled read is what makes the rest of the telemetry meaningful.
WORK WITH PRIVILEGE AI
Know what your agent did, and why.
We build systems that keep a clear record of an agent's actions and the information behind them.
Records built in from the start.
Logging and evidence are part of the design, not an afterthought added later.
Answers when something goes wrong.
When a result needs explaining, the record is already there for your team.
Talk with Privilege AI
Tell us which agent actions must be traceable and which operational signals should bring a person into the workflow.
CONTACT PRIVILEGE AI
Tell us which agent actions must be traceable and which operational signals should bring a person into the workflow.
Privilege AI builds systems where retrieval boundaries, actions and evidence are part of the architecture rather than an afterthought.