The inference host holds nothing.
No standing credentials to data stores, no write access to the corpus. It receives prompts and returns text.
MKC2 results ↗AI infrastructure security is mostly ordinary infrastructure security applied to an environment with unusual dependencies and unusually sensitive logs.
01 / DEFINITION
The short answer first.Secure AI infrastructure is the hardened environment an AI system runs in: segmented networks with controlled egress, verified model artefacts, isolated inference processes, protected secrets and restricted logs. It is the layer beneath model and application security.
Most of this is conventional practice. What makes it worth stating separately for AI is that the dependency surface is unusual — large binary artefacts from public hubs, fast-moving numerical libraries, GPU drivers — and that the logs are unusually sensitive, because they contain the prompts and the documents inside them.
WORKED EXAMPLE
A private model service can have a locked network perimeter and still accept an unverified model package, an overprivileged deployment credential or a retrieval index copied from the wrong project. Secure AI infrastructure requires a chain of controls that covers artefact intake, build, deployment, runtime and recovery. Start with one update and identify who can change each link.
Use a staged release with a known model revision, tokenizer, application image and synthetic retrieval index. Record the expected digests, approved source and identity authorised to promote the release. Verify the artefacts at the point of use. A checksum written in a project document is of little value if the serving host starts whatever file happens to be mounted under the expected name.
Replace the test tokenizer or index with a different harmless artefact and confirm that deployment stops before the service becomes available. Then restore the approved package and verify that the normal release succeeds. Save the decision and the component that enforced it. A deployment pipeline that fails for every package has demonstrated a broken release, not a working integrity control.
Separate the identities used for building, approving and serving. The inference worker may need read access to model weights while having no permission to replace them. A document importer may update an index without being able to grant itself access to another project's store. Test those boundaries using the actual service identities rather than an administrator's personal credentials.
Simulate a model crash and follow its diagnostic output. Determine whether prompts, retrieved passages or credentials enter a crash dump or shared logging system. Give an operator the minimum diagnostic information needed to identify the failed release. Check who can retrieve more sensitive evidence and how that access is recorded.
Finally, recover onto a clean host from approved artefacts and an authorised backup. Confirm the model version, project boundaries and user access after restoration. A backup that restores quickly but grants the service a broader credential recreates the application with a different security posture.
Use the exercise to produce a release manifest, identity map, incident procedure and a set of negative tests. Repeat them when a model format, runtime, storage service or operating team changes. This creates evidence for the particular infrastructure you operate and identifies precisely which component should stop an unauthorised change from reaching users.
02 / THE LAYERS
| Layer | Concern | Control |
|---|---|---|
| Network | Outbound requests from an inference host | Segmentation with a default-deny egress policy and an allowlist |
| Model artefacts | Unverified weights; unsafe serialisation | Checksums, recorded provenance, safe formats, an internal registry |
| Runtime | A compromised dependency reaching the host | Containerisation, non-root execution, minimal images |
| Secrets | Credentials reachable by the inference process | Short-lived tokens from a secret manager; nothing in environment files |
| Logs | Prompts and documents in a widely readable store | Restricted access, redaction where possible, explicit retention |
| Retrieval tier | An index that bypasses source permissions | Per-document access metadata, filtered at query time |
Default-deny egress is the control with the best return, because it closes the exfiltration route that nearly every other AI-specific risk eventually uses: an injected instruction, a compromised dependency and a mistaken tool configuration all need a way out, and an allowlist removes it for all three at once.
03 / THE MODEL SUPPLY CHAIN
Weights downloaded from a public hub with no verification are an unreviewed component with deep access to the environment.
Four practices make this tractable, and none of them is expensive:
The same discipline applies to the numerical and serving libraries around the model, which move quickly and carry substantial native code. Pinning and reviewing them is ordinary supply-chain hygiene applied to an unusually fast-moving ecosystem.
04 / CONTAINMENT
No standing credentials to data stores, no write access to the corpus. It receives prompts and returns text.
Separate processes with separate identities, so a compromise of the generation path does not inherit the index's access.
Caches keyed by identity and scoped per tenant, so a leak cannot propagate between users through an optimisation.
Air-gapping is the strongest form of containment and is worth considering wherever the material justifies it, because it converts a set of controls into a property of the environment: with no outbound route, exfiltration is impossible rather than prevented. MKC2 runs entirely offline for this reason. The trade is that updates, monitoring and support all become scheduled, deliberate activities. See on-premise AI.
05 / QUESTIONS
Default-deny egress from inference and agent hosts. It closes the route that injection, dependency compromise and tool misconfiguration all rely on, and it is cheaper than any of the controls aimed at those individually.
The artefact can be, depending on the serialisation format, and an unverified download is an unreviewed dependency. Verify checksums, prefer safe formats, record provenance and serve from an internal registry rather than pulling from a public hub at deploy time.
Generally yes, for isolation, reproducibility and least privilege, with GPU access passed through. Non-root execution and minimal images do most of the work; the containerisation itself is not the control.
As the most sensitive store in the system, because that is what they are: prompts and retrieved documents in one place. Restrict read access more tightly than the source systems if anything, redact where feasible, and set an explicit retention period.
No. Infrastructure bounds what a compromise can reach; it does nothing about a model producing wrong output or an agent taking a permitted action for a bad reason. Both layers are required. See AI agent security.
WORK WITH PRIVILEGE AI
Secure the whole environment.
We treat the network, information access, permissions and the AI itself as one system.
Layer by layer.
Each part of the environment your AI runs in is hardened.
Private when it matters.
Sensitive work can run inside a boundary you control.
Talk with Privilege AI
Describe the assets your AI infrastructure must protect and which administrator or service boundaries matter most.
CONTACT PRIVILEGE AI
Describe the assets your AI infrastructure must protect and which administrator or service boundaries matter most.
Privilege AI treats security boundaries as part of the system: network, retrieval, permissions and inference designed together.