A single switch that works.
One control that revokes an agent's credentials and stops its runs, testable and tested. If it requires a deployment, it is not a switch.
Someone signs
for what
the agent does.
Agent governance is the set of decisions about who may deploy an agent, what it may do, on whose authority, and how that authority is taken back.
READ THE DEFINITION ↓REGISTRY · TIERS · WITHDRAWAL
01 / DEFINITION
The short answer first.What is AI agent governance?
AI agent governance is the organisational control of autonomous AI systems: which agents exist, what each may access and do, who approved it, what evidence supported the approval, and how autonomy is reduced when the evidence changes. It is the accountability layer above the technical controls.
Governance is often written as policy and then not implemented, which produces documents that describe a system nobody operates. The version that works is closer to change management: a registry of what exists, a defined process for changing it, evidence attached to each decision, and a mechanism that can actually remove a capability.
02 / THE REGISTRY
You cannot govern what you cannot list.The first governance failure is inventory. Agents proliferate faster than most organisations track them, because building one no longer requires a project: a developer with an API key and a scripting environment can create a system that reads production data within an afternoon. A registry that is not backed by a technical control — credential issuance, network egress, tool gateway — records only the agents whose owners chose to register them.
A workable registry entry records, for each agent: its purpose, its owner, the identity it acts as, the tools it may call, the data it may reach, its autonomy tier, the evaluation evidence behind that tier, its review date, and how to switch it off. The last field is the one most often missing and the one most needed at speed.
| Tier | What the agent may do | Evidence required to hold it |
|---|---|---|
| Observe | Read and report. No writes, no external calls. | Access review only |
| Propose | Draft actions for a person to confirm. | Trajectory evaluation showing valid paths |
| Act with logging | Take reversible actions autonomously, fully logged. | Sustained proposal record with measured acceptance |
| Act on the outside | Take actions with external effect. | All of the above, plus an accepted failure cost and a reversal path |
03 / EVIDENCE FOR A GRANT
What should be required before autonomy increases.Promotion needs
a record.
An autonomy increase should be an evidenced decision, not a configuration change made under schedule pressure. The minimum useful evidence set is small and specific:
- A trajectory evaluation on the capability in question, including fault injection and adversarial content, as described in AI agent evaluation.
- A proposal record: how often the agent's proposed actions were accepted by the humans reviewing them, over enough volume to be meaningful.
- A blast-radius statement: the worst outcome of a wrong action at this tier, and whether it is reversible.
- A named accountable owner who carries that risk.
This mirrors how Privilege AI grants authority to supervision checks: MKC2 assigns authority per check by measured ranking quality and calibration, holds 25 of 45 checks at report-only, and refuses one outright. The principle transfers directly — evidence per capability, not a single verdict on the system.
04 / WITHDRAWAL
The part policies usually omit.Automatic demotion triggers.
Defined conditions that drop an agent a tier without a meeting: a spike in rejected proposals, an out-of-scope access attempt, a failed adversarial check.
Standing expiry.
Capabilities that lapse unless renewed with current evidence. Permissions granted for a project routinely outlive it.
Governance also has to name the accountable person for actions taken. An agent is not a legal actor; the organisation deploying it is responsible for what it does. Keeping the agent acting as the requesting user's identity, rather than as a shared service account, preserves that chain in the technical record as well as on paper — see AI agent access control.
05 / QUESTIONS
Asked by the people who sign off on deployment.Who should own AI agent governance?
+
A named owner per agent, with a central function maintaining the registry and the tier definitions. Governance concentrated entirely in a central committee becomes a bottleneck and is routed around; governance distributed with no central record produces no visibility. Both parts are needed.
Do AI agents need separate governance from other AI systems?
+
They need additional governance. A model that produces text is governed as a content risk; an agent that takes actions is also an operational and access risk, with consequences that are not always reversible. The agent-specific part is the authority to act.
How does governance handle third-party agents?
+
The same way as internal ones, with two extra questions: what data leaves the boundary, and what happens when the vendor changes the model behind the service. The second is a real operational risk, since behaviour can change without any change on your side. See AI model monitoring.
What records will a governance review ask for?
+
Typically: what the agent does, what it can access, who approved it, what evidence supported the approval, what it has actually done, and what happens if it malfunctions. A registry with evaluation evidence and trajectory logs answers all six; a policy document answers none.
How often should agent permissions be reviewed?
+
On a fixed cycle and on every capability change. Expiring grants by default is more reliable than scheduled reviews, because an expiry forces a decision while a review can be deferred indefinitely. See AI agent least privilege.
Grant authority
on evidence.
Privilege AI works on the measurement that governance decisions should rest on: trajectory evaluation, supervision and per-capability authority.