Choose one bounded task.
A task with a clear input, a clear output and a person who currently does it. Platform-first programmes produce capability with no owner.
One task.
Measured first.
Private AI deployments stall for predictable reasons. The sequence that avoids most of them starts with a narrow task and an evaluation set, not with a platform.
READ THE SEQUENCE ↓SCOPE · MEASURE · REVIEW · EXTEND
01 / DEFINITION
The short answer first.What does private AI deployment involve?
Private AI deployment is the process of putting a model into production inside an organisation's own boundary: choosing the task, building the evaluation, provisioning inference, integrating with systems of record, setting the authority the system holds, and operating it. The order matters more than the tooling.
Most stalled AI projects did not fail technically. They failed because nobody could say whether the system was good enough, so the decision to extend or withdraw it was never made. An evaluation set built before deployment is the artefact that prevents this, and it costs a fraction of the integration work.
02 / THE SEQUENCE
Four phases, in this order.Build the evaluation set.
Held out, representative, with the standard written down by the expert who owns the work. This is the acceptance criterion and the baseline.
Deploy in review-only mode.
The system reports or proposes; a person decides. Real inputs, real workflow, bounded consequences, and a disposition record accumulating.
Extend on the record.
More authority, or a second task, justified by measured dispositions rather than by enthusiasm from the pilot.
Phase three is where the useful evidence is generated, and it is the phase most often skipped in favour of going straight to automation. Review-only deployment produces something no benchmark can: a record of how the system behaves on live inputs, with human dispositions attached. That record is what makes the phase-four decision defensible.
Privilege AI applies the same ceiling to its own supervision model. MKC2 runs in review-only mode — it reports and flags, it does not act — and its published record states plainly that no attorney dispositions have yet been recorded, so the figures describe held-out evaluation rather than agreement on live work. See MKC2.
03 / WHAT TO DECIDE BEFORE THE BUILD
Six decisions that are expensive to revisit.| Decision | Why it is hard to change later |
|---|---|
| Where inference runs | Determines the data boundary, the hardware and the contracts |
| Whose identity retrieval uses | Retrofitting per-user filtering means rebuilding the index |
| What the system is allowed to do | Adding a human review step after automation is a workflow change |
| What is logged, and who can read it | Logs accumulate; a permissive early choice is hard to unwind |
| How the model version is pinned | Without pinning, no result can be reproduced or attributed |
| Who owns the evaluation standard | Without a named owner the criteria drift and results stop comparing |
The second row is the one that most often forces a rebuild. An index constructed without per-document access metadata cannot be made identity-aware by configuration; it has to be rebuilt with permissions carried through the pipeline. Deciding this on day one costs nothing. See RAG access control.
04 / FAILURE PATTERNS
How private AI projects actually stall.The impressive demo.
Built on cherry-picked inputs with no evaluation set, so the gap between demonstration and production is discovered only after commitment.
The platform without a task.
Infrastructure delivered before any use case owns it. Capability with no consumer generates cost and no evidence.
Automation before evidence.
Autonomy granted on pilot enthusiasm rather than a disposition record, which makes the first visible failure a governance problem.
A fourth pattern is subtler: the unowned evaluation. An engineering team builds the evaluation set because the domain experts are busy, the criteria drift towards what is easy to measure, and the eventual score does not persuade the people who have to rely on the system. The standard has to come from whoever owns the work, which is the argument in AI evaluation framework.
05 / QUESTIONS
Asked when planning a first deployment.How long does a private AI deployment take?
+
The variable parts are data access, integration and the evaluation set — not the inference stack, which is now routine. A narrow task with existing data access and a willing expert owner moves quickly; one requiring new integrations and a new labelled corpus takes considerably longer, and that work is the deployment rather than a precursor to it.
Should we start with a platform or a use case?
+
A use case. Platform-first programmes produce infrastructure nobody has committed to using, and the second use case is usually easier to add than the first regardless of how general the platform was. Generality earned from two real tasks beats generality designed in advance.
Do we need our own model, or can we use an open-weight one?
+
Start with an open-weight model and prompting, and measure. Fine-tune only when the measurement shows the failure is about judgement rather than information. See AI model fine-tuning.
What does review-only mode mean in practice?
+
The system produces findings or draft actions and a person decides. It gives you real inputs, real workflow integration and an accumulating disposition record, with a worst case of wasted review time rather than a wrong outcome.
When is it right to increase autonomy?
+
When the disposition record over meaningful volume supports it, the blast radius is bounded, and a named owner accepts the residual risk. Autonomy granted on a pilot impression rather than a record is the most common source of the first visible failure.
Start with one task
and a measurement.
Privilege AI works across model adaptation, retrieval and private deployment, and builds the evaluation before the rollout.