Privilege AIMKC2 results ↗

AI agent governance

Agent governance is the set of decisions about who may deploy an agent, what it may do, on whose authority, and how that authority is taken back.

READ THE DEFINITION ↓REGISTRY · TIERS · WITHDRAWAL

01 / DEFINITION

The short answer first.

What is AI agent governance?

AI agent governance is the organisational control of autonomous AI systems: which agents exist, what each may access and do, who approved it, what evidence supported the approval, and how autonomy is reduced when the evidence changes. It is the accountability layer above the technical controls.

Governance is often written as policy and then not implemented, which produces documents that describe a system nobody operates. The version that works is closer to change management: a registry of what exists, a defined process for changing it, evidence attached to each decision, and a mechanism that can actually remove a capability.

WORKED EXAMPLE

Decide who can change an agent's purchasing authority

A purchasing assistant starts as a research tool that compares suppliers. A team then proposes allowing it to submit orders below a spending limit. That is a change in authority, even if the model, interface and supplier list remain the same. Governance should identify who approves the new action, which evidence they review and who can withdraw permission if the arrangement fails.

Write the decision in terms of an actual order

Choose a routine replenishment order and specify the requestor, permitted supplier, item category, maximum total and approval expiry. Identify whether the limit applies per order or across an entire period. Several individually small orders can exceed a budget that was intended to cover a month. Put the business meaning of the limit in the approval record so engineering does not have to infer it from a single number.

Ask the policy owner how to handle a supplier substitution, a split shipment and an order that changes after approval. Run each case in a test environment. A manager's approval of a particular quantity and price should not silently authorise a different quantity or a new destination. Save the proposed transaction alongside the decision and the identity of the approving person.

Rehearse withdrawal as well as approval

Remove purchasing authority while an order is queued. Confirm that the tool checks the current policy at execution and that a previously issued token cannot complete a newly forbidden purchase. The agent may still be allowed to research alternatives or draft a request for a person.

Review the audit record afterward. An operator should be able to identify which policy version was active, what the agent attempted, who approved the specific transaction and what the supplier system actually accepted. That evidence makes the governance decision reviewable when the business next considers increasing the spending limit.

02 / THE REGISTRY

The first governance failure is inventory. Agents proliferate faster than most organisations track them, because building one no longer requires a project: a developer with an API key and a scripting environment can create a system that reads production data within an afternoon. A registry that is not backed by a technical control — credential issuance, network egress, tool gateway — records only the agents whose owners chose to register them.

A workable registry entry records, for each agent: its purpose, its owner, the identity it acts as, the tools it may call, the data it may reach, its autonomy tier, the evaluation evidence behind that tier, its review date, and how to switch it off. The last field is the one most often missing and the one most needed at speed.

Autonomy tiers. An agent moves up a tier on evidence and down a tier on incident, and the route down should be faster than the route up.
TierWhat the agent may doEvidence required to hold it
ObserveRead and report. No writes, no external calls.Access review only
ProposeDraft actions for a person to confirm.Trajectory evaluation showing valid paths
Act with loggingTake reversible actions autonomously, fully logged.Sustained proposal record with measured acceptance
Act on the outsideTake actions with external effect.All of the above, plus an accepted failure cost and a reversal path

03 / EVIDENCE FOR A GRANT

Promotion needs
a record.

What should be required before autonomy increases.

An autonomy increase should be an evidenced decision, not a configuration change made under schedule pressure. The minimum useful evidence set is small and specific:

  • A trajectory evaluation on the capability in question, including fault injection and adversarial content, as described in AI agent evaluation.
  • A proposal record: how often the agent's proposed actions were accepted by the humans reviewing them, over enough volume to be meaningful.
  • A blast-radius statement: the worst outcome of a wrong action at this tier, and whether it is reversible.
  • A named accountable owner who carries that risk.

This mirrors how Privilege AI grants authority to supervision checks: MKC2 assigns authority per check by measured ranking quality and calibration, holds 25 of 45 checks at report-only, and refuses one outright. The principle transfers directly — evidence per capability, not a single verdict on the system.

04 / WITHDRAWAL

↳ 01

A single switch that works.

One control that revokes an agent's credentials and stops its runs, testable and tested. If it requires a deployment, it is not a switch.

↳ 02

Automatic demotion triggers.

Defined conditions that drop an agent a tier without a meeting: a spike in rejected proposals, an out-of-scope access attempt, a failed adversarial check.

↳ 03

Standing expiry.

Capabilities that lapse unless renewed with current evidence. Permissions granted for a project routinely outlive it.

Governance also has to name the accountable person for actions taken. An agent is not a legal actor; the organisation deploying it is responsible for what it does. Keeping the agent acting as the requesting user's identity, rather than as a shared service account, preserves that chain in the technical record as well as on paper — see AI agent access control.

05 / QUESTIONS

Who should own AI agent governance?

A named owner per agent, with a central function maintaining the registry and the tier definitions. Governance concentrated entirely in a central committee becomes a bottleneck and is routed around; governance distributed with no central record produces no visibility. Both parts are needed.

Do AI agents need separate governance from other AI systems?

They need additional governance. A model that produces text is governed as a content risk; an agent that takes actions is also an operational and access risk, with consequences that are not always reversible. The agent-specific part is the authority to act.

How does governance handle third-party agents?

The same way as internal ones, with two extra questions: what data leaves the boundary, and what happens when the vendor changes the model behind the service. The second is a real operational risk, since behaviour can change without any change on your side. See AI model monitoring.

What records will a governance review ask for?

Typically: what the agent does, what it can access, who approved it, what evidence supported the approval, what it has actually done, and what happens if it malfunctions. A registry with evaluation evidence and trajectory logs answers all six; a policy document answers none.

How often should agent permissions be reviewed?

On a fixed cycle and on every capability change. Expiring grants by default is more reliable than scheduled reviews, because an expiry forces a decision while a review can be deferred indefinitely. See AI agent least privilege.

WORK WITH PRIVILEGE AI

How we can help

Talk with Privilege AI

Tell us who currently owns agent permissions, release approval and the right to stop a running workflow.

Talk to us →

CONTACT PRIVILEGE AI

Contact Us

Tell us who currently owns agent permissions, release approval and the right to stop a running workflow.

When should we call?

By submitting, you agree that Privilege AI, a division of Tech1e LLC, may call or email you about this inquiry. Please leave confidential information out of your message.

Grant authority
on evidence.

Privilege AI works on the measurement that governance decisions should rest on: trajectory evaluation, supervision and per-capability authority.