Privilege AIMKC2 results ↗

Secure AI infrastructure

AI infrastructure security is mostly ordinary infrastructure security applied to an environment with unusual dependencies and unusually sensitive logs.

READ THE LAYERS ↓SEGMENT · VERIFY · CONTAIN

01 / DEFINITION

The short answer first.

What is secure AI infrastructure?

Secure AI infrastructure is the hardened environment an AI system runs in: segmented networks with controlled egress, verified model artefacts, isolated inference processes, protected secrets and restricted logs. It is the layer beneath model and application security.

Most of this is conventional practice. What makes it worth stating separately for AI is that the dependency surface is unusual — large binary artefacts from public hubs, fast-moving numerical libraries, GPU drivers — and that the logs are unusually sensitive, because they contain the prompts and the documents inside them.

WORKED EXAMPLE

Follow a model update from supplier to serving host

A private model service can have a locked network perimeter and still accept an unverified model package, an overprivileged deployment credential or a retrieval index copied from the wrong project. Secure AI infrastructure requires a chain of controls that covers artefact intake, build, deployment, runtime and recovery. Start with one update and identify who can change each link.

Use a staged release with a known model revision, tokenizer, application image and synthetic retrieval index. Record the expected digests, approved source and identity authorised to promote the release. Verify the artefacts at the point of use. A checksum written in a project document is of little value if the serving host starts whatever file happens to be mounted under the expected name.

Attempt a substitution in a disposable environment

Replace the test tokenizer or index with a different harmless artefact and confirm that deployment stops before the service becomes available. Then restore the approved package and verify that the normal release succeeds. Save the decision and the component that enforced it. A deployment pipeline that fails for every package has demonstrated a broken release, not a working integrity control.

Separate the identities used for building, approving and serving. The inference worker may need read access to model weights while having no permission to replace them. A document importer may update an index without being able to grant itself access to another project's store. Test those boundaries using the actual service identities rather than an administrator's personal credentials.

Inspect the paths that appear during an incident

Simulate a model crash and follow its diagnostic output. Determine whether prompts, retrieved passages or credentials enter a crash dump or shared logging system. Give an operator the minimum diagnostic information needed to identify the failed release. Check who can retrieve more sensitive evidence and how that access is recorded.

Finally, recover onto a clean host from approved artefacts and an authorised backup. Confirm the model version, project boundaries and user access after restoration. A backup that restores quickly but grants the service a broader credential recreates the application with a different security posture.

Use the exercise to produce a release manifest, identity map, incident procedure and a set of negative tests. Repeat them when a model format, runtime, storage service or operating team changes. This creates evidence for the particular infrastructure you operate and identifies precisely which component should stop an unauthorised change from reaching users.

02 / THE LAYERS

Infrastructure layers, the AI-specific concern at each, and the control. The first row is the highest-value single control in the list.
LayerConcernControl
NetworkOutbound requests from an inference hostSegmentation with a default-deny egress policy and an allowlist
Model artefactsUnverified weights; unsafe serialisationChecksums, recorded provenance, safe formats, an internal registry
RuntimeA compromised dependency reaching the hostContainerisation, non-root execution, minimal images
SecretsCredentials reachable by the inference processShort-lived tokens from a secret manager; nothing in environment files
LogsPrompts and documents in a widely readable storeRestricted access, redaction where possible, explicit retention
Retrieval tierAn index that bypasses source permissionsPer-document access metadata, filtered at query time

Default-deny egress is the control with the best return, because it closes the exfiltration route that nearly every other AI-specific risk eventually uses: an injected instruction, a compromised dependency and a mistaken tool configuration all need a way out, and an allowlist removes it for all three at once.

03 / THE MODEL SUPPLY CHAIN

A model file
is a dependency.

Treating weights as a dependency.

Weights downloaded from a public hub with no verification are an unreviewed component with deep access to the environment.

Four practices make this tractable, and none of them is expensive:

  • An internal registry. Models enter once, through a controlled import, and are served from inside. Production hosts do not download from the internet.
  • Checksums and recorded provenance. Which file, from where, verified when, by whom. This is also what makes an evaluation result attributable years later.
  • Safe serialisation only. Some formats permit arbitrary code execution on load. Where a conversion is needed, do it once in an isolated environment.
  • Pinned versions everywhere. A model referenced by a moving tag is a dependency that can change without a deployment.

The same discipline applies to the numerical and serving libraries around the model, which move quickly and carry substantial native code. Pinning and reviewing them is ordinary supply-chain hygiene applied to an unusually fast-moving ecosystem.

04 / CONTAINMENT

↳ 01

The inference host holds nothing.

No standing credentials to data stores, no write access to the corpus. It receives prompts and returns text.

↳ 02

Retrieval and generation are separate.

Separate processes with separate identities, so a compromise of the generation path does not inherit the index's access.

↳ 03

Nothing shares a cache.

Caches keyed by identity and scoped per tenant, so a leak cannot propagate between users through an optimisation.

Air-gapping is the strongest form of containment and is worth considering wherever the material justifies it, because it converts a set of controls into a property of the environment: with no outbound route, exfiltration is impossible rather than prevented. MKC2 runs entirely offline for this reason. The trade is that updates, monitoring and support all become scheduled, deliberate activities. See on-premise AI.

05 / QUESTIONS

What is the single most valuable AI infrastructure control?

Default-deny egress from inference and agent hosts. It closes the route that injection, dependency compromise and tool misconfiguration all rely on, and it is cheaper than any of the controls aimed at those individually.

Are model weights a security risk?

The artefact can be, depending on the serialisation format, and an unverified download is an unreviewed dependency. Verify checksums, prefer safe formats, record provenance and serve from an internal registry rather than pulling from a public hub at deploy time.

Should inference run in containers?

Generally yes, for isolation, reproducibility and least privilege, with GPU access passed through. Non-root execution and minimal images do most of the work; the containerisation itself is not the control.

How should AI logs be protected?

As the most sensitive store in the system, because that is what they are: prompts and retrieved documents in one place. Restrict read access more tightly than the source systems if anything, redact where feasible, and set an explicit retention period.

Does secure infrastructure remove the need for model-level controls?

No. Infrastructure bounds what a compromise can reach; it does nothing about a model producing wrong output or an agent taking a permitted action for a bad reason. Both layers are required. See AI agent security.

WORK WITH PRIVILEGE AI

How we can help

Talk with Privilege AI

Describe the assets your AI infrastructure must protect and which administrator or service boundaries matter most.

Talk to us →

CONTACT PRIVILEGE AI

Contact Us

Describe the assets your AI infrastructure must protect and which administrator or service boundaries matter most.

When should we call?

By submitting, you agree that Privilege AI, a division of Tech1e LLC, may call or email you about this inquiry. Please leave confidential information out of your message.

Build the room
around the model.

Privilege AI treats security boundaries as part of the system: network, retrieval, permissions and inference designed together.