Bound what one run can touch.
Per-session limits on records read, actions taken and volume moved. An agent that can legitimately read one matter should not be able to read ten thousand in an afternoon.
The instruction
arrives with
the data.
An AI agent reads untrusted content and holds real credentials. AI agent security is the practice of assuming those two facts will eventually meet.
READ THE THREAT MODEL ↓PRIVILEGE · INJECTION · BLAST RADIUS
01 / DEFINITION
The short answer first.What is AI agent security?
AI agent security is the discipline of controlling what an autonomous AI system can access and do. It differs from application security because an agent's instructions and its input arrive through the same channel: any document, email or web page the agent reads is a potential instruction it may follow.
Conventional software separates code from data. A parser reading a malicious file may crash, but it does not take the file's contents as a command. Language models have no such separation. The system prompt, the user's request and the retrieved document are one token stream, and the model weighs them by plausibility rather than by origin.
That single property generates most of the agent threat model. Everything else — tool scoping, credential handling, audit — is an attempt to limit what happens when the model follows the wrong instruction, because preventing it entirely is not currently achievable.
02 / THE THREAT MODEL
What is actually different about agents.| Threat | How it works | Primary control |
|---|---|---|
| Indirect prompt injection | Instructions hidden in content the agent reads | Treat all retrieved content as untrusted; constrain tools |
| Confused deputy | The agent uses its own privileges on behalf of a less privileged user | Act as the user's identity, not the agent's |
| Excessive scope | A credential broader than the task requires | Per-task scoping and short-lived tokens |
| Data exfiltration | Sensitive context leaves via a tool call, URL or webhook | Egress allowlists; no free-form outbound requests |
| Chained escalation | Low-risk tools combine into a high-risk capability | Review tool combinations, not individual tools |
| Trajectory drift | A long-running loop wanders from the original objective | Step limits, objective checks, human checkpoints |
Chained escalation is the threat that formal reviews miss most often, because each tool is approved in isolation. Read-a-file and send-an-email are both individually unremarkable; together they are an exfiltration path. The unit of security review has to be the capability set, not the tool. That analysis is developed in AI agent privilege escalation.
03 / CONTROLS THAT HOLD
Where to put the enforcement.Enforce outside
the model.
A control implemented in a prompt is a request. A control implemented in the tool layer is a rule. Only the second survives an instruction the model decides to follow.
The practical consequence is architectural. Every restriction that matters — which records are readable, which actions are permitted, which destinations are reachable — belongs in code between the model and the resource, evaluated per call, with the requesting user's identity attached. Prompt-level instructions such as never reveal the system prompt or only use information from the provided documents are useful for shaping ordinary behaviour and provide no guarantee under adversarial input.
Four controls carry most of the weight in practice:
- Identity propagation. The agent acts as the user, so existing permissions apply unchanged and the agent cannot become a privilege amplifier.
- Per-call authorisation. Every tool invocation is checked at the moment of use, not once at session start.
- Egress control. Outbound network access restricted to an allowlist, which closes the most direct exfiltration route.
- Review-only defaults. New capabilities start as proposals a human confirms, and earn autonomy from a measured record.
04 / BLAST RADIUS
Designing for the day it goes wrong.Make everything reversible where you can.
Drafts instead of sends, staged changes instead of commits, soft deletes. Reversibility is worth more than confidence in the model.
Log the intent, not just the call.
A log of tool invocations without the reasoning and the triggering content cannot answer why an action happened, which is the only question that matters after an incident.
Bounding the blast radius is also what makes measured autonomy possible. An agent whose worst case is a queue of proposals a human rejects can be given room to fail and be evaluated on a real record; one whose worst case is an irreversible external action cannot. The evaluation side of that argument is in AI agent evaluation, and the organisational side in AI agent governance.
05 / QUESTIONS
Asked before an agent is given credentials.Can prompt injection be prevented?
+
Not reliably with current models. Input filtering, instruction hierarchies and guard models raise the cost of an attack without closing the channel, because the underlying architecture has no way to mark tokens as non-instructions. Designs that assume containment rather than prevention are the ones that hold. See AI agent prompt injection.
Is a read-only agent safe?
+
Safer, not safe. A read-only agent can still exfiltrate whatever it reads if it has any outbound channel, including a URL it can render or fetch. Read-only removes the write risk and leaves the disclosure risk, which is the dominant one for confidential material.
Should agents have their own service accounts?
+
Generally no. A shared service account with broad rights turns the agent into a privilege amplifier and destroys attribution. Propagating the requesting user's identity keeps existing access controls meaningful and keeps the audit trail attributable.
Does running an agent on-premise make it secure?
+
It removes one class of risk — third-party data exposure — and leaves the agent threat model intact. An on-premise agent with broad credentials and a reachable network is still an injection target. Private deployment and agent security are complementary. See private AI.
How do you test agent security?
+
With adversarial cases in the regression suite: documents containing injected instructions, tool responses that lie, contradictory sources, and inputs designed to trigger tool chains. The suite belongs in continuous testing rather than in an annual review. See AI model testing.
Bound what
the agent can do.
Privilege AI works on permissions, retrieval boundaries and private deployment for systems that act on real data.