Choose one bounded task.
A task with a clear input, a clear output and a person who currently does it. Platform-first programmes produce capability with no owner.
MKC2 results ↗Private AI deployments stall for predictable reasons. The sequence that avoids most of them starts with a narrow task and an evaluation set, not with a platform.
01 / DEFINITION
The short answer first.Private AI deployment is the process of putting a model into production inside an organisation's own boundary: choosing the task, building the evaluation, provisioning inference, integrating with systems of record, setting the authority the system holds, and operating it. The order matters more than the tooling.
Most stalled AI projects did not fail technically. They failed because nobody could say whether the system was good enough, so the decision to extend or withdraw it was never made. An evaluation set built before deployment is the artefact that prevents this, and it costs a fraction of the integration work.
WORKED EXAMPLE
A private deployment is ready for a more demanding test when a new operator can install it from the approved artefacts and perform the intended task with outbound network access disabled. A workstation that succeeds only because its developer account already cached a model, tokenizer or dependency is a weak acceptance environment. Start with a clean host that resembles the actual deployment target.
Identify the model weights, tokenizer, application image, retrieval index, configuration and compatible hardware runtime. Record their versions and checksums. Include the licence and update source where relevant to the operator's process. Test the installed bundle rather than the build engineer's machine. A missing tokenizer file can otherwise appear as an unexpected download attempt during the first real request.
Load a synthetic document, execute a representative query and inspect the source returned to the authorised user. Test a second user without document access. Then restart the service and confirm that the permission boundary, model version and persistent index survive the restart. A successful first response does not prove that recovery works.
Introduce a candidate that deliberately produces an incompatible field in a harmless test fixture. Confirm that monitoring detects the problem and that the operator can restore the previous application, model and compatible index together. Record the time required and whether user-visible work was lost or duplicated. If an index migration cannot be reversed, document how to restore its matching snapshot.
These checks give the deployment owner a concrete signoff record covering offline operation, authorised retrieval and recovery. They also reveal which dependencies must be delivered through the controlled update process before the system moves from a demonstration into regular use.
02 / THE SEQUENCE
A task with a clear input, a clear output and a person who currently does it. Platform-first programmes produce capability with no owner.
Held out, representative, with the standard written down by the expert who owns the work. This is the acceptance criterion and the baseline.
The system reports or proposes; a person decides. Real inputs, real workflow, bounded consequences, and a disposition record accumulating.
More authority, or a second task, justified by measured dispositions rather than by enthusiasm from the pilot.
Phase three is where the useful evidence is generated, and it is the phase most often skipped in favour of going straight to automation. Review-only deployment produces something no benchmark can: a record of how the system behaves on live inputs, with human dispositions attached. That record is what makes the phase-four decision defensible.
Privilege AI applies the same ceiling to its own supervision model. MKC2 runs in review-only mode — it reports and flags, it does not act — and its published record states plainly that no attorney dispositions have yet been recorded, so the figures describe held-out evaluation rather than agreement on live work. See MKC2.
03 / WHAT TO DECIDE BEFORE THE BUILD
| Decision | Why it is hard to change later |
|---|---|
| Where inference runs | Determines the data boundary, the hardware and the contracts |
| Whose identity retrieval uses | Retrofitting per-user filtering means rebuilding the index |
| What the system is allowed to do | Adding a human review step after automation is a workflow change |
| What is logged, and who can read it | Logs accumulate; a permissive early choice is hard to unwind |
| How the model version is pinned | Without pinning, no result can be reproduced or attributed |
| Who owns the evaluation standard | Without a named owner the criteria drift and results stop comparing |
The second row is the one that most often forces a rebuild. An index constructed without per-document access metadata cannot be made identity-aware by configuration; it has to be rebuilt with permissions carried through the pipeline. Deciding this on day one costs nothing. See RAG access control.
04 / FAILURE PATTERNS
Built on cherry-picked inputs with no evaluation set, so the gap between demonstration and production is discovered only after commitment.
Infrastructure delivered before any use case owns it. Capability with no consumer generates cost and no evidence.
Autonomy granted on pilot enthusiasm rather than a disposition record, which makes the first visible failure a governance problem.
A fourth pattern is subtler: the unowned evaluation. An engineering team builds the evaluation set because the domain experts are busy, the criteria drift towards what is easy to measure, and the eventual score does not persuade the people who have to rely on the system. The standard has to come from whoever owns the work, which is the argument in AI evaluation framework.
05 / QUESTIONS
The variable parts are data access, integration and the evaluation set — not the inference stack, which is now routine. A narrow task with existing data access and a willing expert owner moves quickly; one requiring new integrations and a new labelled corpus takes considerably longer, and that work is the deployment rather than a precursor to it.
A use case. Platform-first programmes produce infrastructure nobody has committed to using, and the second use case is usually easier to add than the first regardless of how general the platform was. Generality earned from two real tasks beats generality designed in advance.
Start with an open-weight model and prompting, and measure. Fine-tune only when the measurement shows the failure is about judgement rather than information. See AI model fine-tuning.
The system produces findings or draft actions and a person decides. It gives you real inputs, real workflow integration and an accumulating disposition record, with a worst case of wasted review time rather than a wrong outcome.
When the disposition record over meaningful volume supports it, the blast radius is bounded, and a named owner accepts the residual risk. Autonomy granted on a pilot impression rather than a record is the most common source of the first visible failure.
WORK WITH PRIVILEGE AI
Start small, then grow.
We begin with one task and a clear measure of success before a wider rollout.
Everything in one place.
We cover adapting the model, connecting your information and deploying it privately.
Testing before rollout.
We build the evaluation before the system goes live.
Talk with Privilege AI
Tell us where the application must run, how updates enter that environment and what would trigger a rollback.
CONTACT PRIVILEGE AI
Tell us where the application must run, how updates enter that environment and what would trigger a rollback.
Privilege AI works across model adaptation, retrieval and private deployment, and builds the evaluation before the rollout.