Hosted inference.
Every prompt and document reaches a third party. The control is contractual and configurational, and the behaviour of the model can change without any change on your side.
New surface,
old discipline.
Most of securing an AI system is ordinary security done properly. The remainder is genuinely new, and it is where deployments are actually failing.
READ THE SURFACE MODEL ↓SUPPLY CHAIN · CONTEXT · INFERENCE
01 / DEFINITION
The short answer first.What is AI security?
AI security is the protection of systems built on machine learning models, covering the model supply chain, the data that reaches the model at inference time, the trust boundary inside the context window, and the actions the system can take. It extends application security rather than replacing it.
It is worth separating the genuinely new problems from the familiar ones, because conflating them produces programmes that buy novel tooling while leaving basic controls unimplemented. Authentication, network segmentation, secrets management, patching and logging are unchanged and remain the majority of the work.
What is new is narrower and sharper: a model is an opaque artefact from a supply chain, its input channel carries instructions it cannot distinguish from data, and its context window accumulates material from sources with different trust levels.
02 / THE FIVE LAYERS
Where AI-specific risk actually sits.| Layer | AI-specific exposure | Primary control |
|---|---|---|
| Model artefact | Unverified weights; unsafe serialisation formats; unclear provenance | Checksums, safe formats, a recorded source for every model |
| Training data | Sensitive material memorised; poisoned examples | Curated pipelines; access control on the corpus; memorisation testing |
| Inference input | Injected instructions in retrieved content | Untrusted-by-default handling; constrained tools |
| Context window | Material from several trust levels combined in one prompt | Identity-filtered retrieval; no cross-tenant context |
| Output and actions | Exfiltration through generated URLs; unauthorised side effects | Egress allowlists; output sanitisation; per-call authorisation |
The model artefact layer is the one most often skipped, and it is conventional supply-chain security applied to a new object. Model weights are executable in effect: some serialisation formats permit arbitrary code on load, and a model downloaded from a public hub with no verification is an unreviewed dependency with unusually deep access. Recording provenance and pinning versions costs very little.
03 / THE CONTEXT WINDOW IS A TRUST BOUNDARY
The idea most missing from AI security reviews.One prompt,
several trust levels.
A context window routinely contains a system prompt, a user request, internal documents and third-party content — with no mechanism inside the model for keeping them apart.
Treating the context window as a trust boundary changes design decisions. It means asking, for every deployment, which sources can contribute tokens, who controls each of them, and what the most privileged action available to the resulting response is. Where a low-trust source can contribute to a context that has access to a high-privilege action, that is an architectural finding regardless of how the prompt is worded.
The practical response is separation: a component that reads untrusted material produces a structured, validated result, and a separate component holds any privileged capability. The interface between them is a schema rather than a model's judgement, which is the pattern described in AI agent privilege escalation.
04 / WHERE DEPLOYMENT CHANGES THE ANSWER
What the hosting decision does to the surface.Private inference.
The material stays inside a boundary you operate, and the version changes when you decide. In exchange you own the infrastructure, patching and capacity.
Either way, the agent surface remains.
Injection, excessive scope and exfiltration are properties of the architecture, not of where the weights run.
Privilege AI works on the private side of that choice: local model execution with explicit hardware, network and operational constraints, and permissions and retrieval boundaries that define which information a model can access and which actions it can propose. Where material is confidential by default — privileged legal documents being the clearest case — the hosting decision is usually settled by that fact alone. See private AI and AI data security.
05 / QUESTIONS
Asked at the start of an AI security review.Is AI security different from application security?
+
It is an extension. The infrastructure, identity and network controls are the same and still do most of the work. What is added is a model supply chain, an input channel that carries instructions, and a context window that mixes trust levels — three things conventional reviews have no question for.
What is the biggest practical AI security risk today?
+
For systems that act, the combination of untrusted input and broad capability: an agent that reads third-party content and holds credentials wide enough to matter. For systems that only generate text, it is data exposure through the inference path.
Do AI systems need penetration testing?
+
Yes, with an expanded scope: injection through every content channel, tool-chain composition, egress paths, and retrieval boundary bypass alongside conventional application testing. A test that only exercises the chat interface misses most of the surface.
Does an on-premise deployment remove the AI security problem?
+
It removes third-party data exposure and gives you version control. Injection, excessive privilege and exfiltration through legitimate features are unaffected, because they are architectural. See secure AI infrastructure.
Who should own AI security?
+
Existing security functions, with AI-specific competence added, rather than a separate team. Splitting it creates a gap at exactly the boundary where problems live: between the model and the systems it reaches.
Secure the layer
that is actually new.
Privilege AI works on private deployment, retrieval boundaries and the controls that decide what a model can reach.