Hardware.
For a specialised 8B-class model at practice scale, a single modern GPU with headroom for key-value cache at realistic document lengths.
Privileged material
stays inside.
For a practice, the question is not whether a model is capable. It is whether client material has to leave the building for the model to read it.
READ THE DEFINITION ↓OFFLINE · MATTER-LEVEL · BOUNDED
01 / DEFINITION
The short answer first.What is private legal AI?
Private legal AI is the deployment of AI models on infrastructure a practice controls, so that privileged and confidential client material is never transmitted to a third-party inference service. Prompts, documents and outputs remain inside the practice's own boundary.
The argument is simpler here than in most industries. Client material is confidential by default, and the strongest statement a practice can make about it is that it did not leave. That is a property of the architecture rather than of a contract, and it is verifiable: a reviewer can enumerate the network destinations the inference path can reach.
MKC2 was built this way. It runs entirely offline, reviewing AI-generated legal drafting and the actions of legal AI agents without any external inference call. See MKC2.
02 / WHAT A PRACTICE ACTUALLY NEEDS
Beyond the model itself.| Component | Requirement specific to legal work |
|---|---|
| Inference | Local execution, no external calls, pinned version |
| Retrieval | Matter-level and client-level access filtering at query time |
| Conflicts and walls | Hard separation that a query filter cannot accidentally omit |
| Provenance | Every retrieved passage traceable to a document and version |
| Authority | Review-only by default; no action on a live matter |
| Records | What was asked, what was retrieved, what was produced |
Matter-level access control is the requirement that most often forces an index rebuild. Retrieval has to be filtered by the requesting person's entitlement to that matter, applied before relevance scoring, with permission changes taking effect immediately. An index that reflects last night's permissions is enforcing last night's ethical walls. See RAG access control.
Where the separation is a hard boundary rather than a preference, partitioned indexes are stronger than filters, because isolation becomes structural rather than dependent on a filter being applied correctly on every query path.
03 / WHAT IT COSTS TO RUN
The operational side, stated plainly.Operations.
Capacity monitoring, driver and firmware compatibility, a controlled update route for an isolated network, and someone available for hardware failure.
Evaluation.
A held-out set drawn from the practice's own material, with the standard written by the practitioners who own the work.
Review workflow.
A queue ordered by calibrated confidence, and a recorded disposition for every flag, which is what eventually measures the system on live work.
Step three is the one that determines whether the deployment is trusted. Vendor figures on public datasets say little about performance on one practice's documents, vocabulary and definition of a defect. Building the set is the work that converts an interesting system into one practitioners rely on, and it belongs to the practitioners rather than to engineering. See legal AI evaluation.
Step four is the route to evidence nobody currently has. Privilege AI states plainly that no attorney dispositions have been recorded for MKC2, so its published figures describe held-out evaluation rather than agreement on live matters. A disposition record is how that gap closes.
04 / WHAT PRIVATE DEPLOYMENT DOES NOT SETTLE
Remaining obligations.Access inside the practice.
A local system with an unpermissioned index still lets anyone read every matter. Private and access-controlled are different properties.
Professional duties.
Competence, confidentiality and supervision rest with the practice. A system can support them; it cannot discharge them.
Verification of output.
Citations and propositions still require checking. The system's job is to direct that checking, not to remove it.
Rules of professional conduct and any client-specific undertakings differ by jurisdiction and engagement, and are a matter for the practice. What a technology supplier can legitimately offer is a system whose data boundary, measured performance and authority limits are precise enough to be assessed against those rules — which is the standard Privilege AI holds its own published work to.
05 / QUESTIONS
Asked by practices considering local deployment.Does using AI on client documents require them to leave the firm?
+
No. Open-weight models fine-tuned for specific tasks run on local hardware, and a well-chosen specialised model can outperform a larger general one on that task. MKC2 runs entirely offline.
Is a vendor's no-training commitment sufficient?
+
It is a meaningful assurance about provider behaviour, not a change in where processing happens. Whether that distinction matters is a judgement for the practice, and for material that is confidential by default it usually decides the architecture on its own.
What hardware does a practice need?
+
For a specialised 8B-class model with moderate concurrency, a single modern GPU with enough memory for weights plus cache at realistic document lengths. The sizing arithmetic is in private AI infrastructure.
How are ethical walls handled?
+
As hard filters applied at query time from the authoritative source, and preferably as partitioned indexes so the isolation is structural. A nightly permission synchronisation is not adequate for a boundary that exists to prevent a specific person seeing a specific matter.
Can a private legal AI system be audited?
+
That is one of its advantages. Every component is inside the practice's boundary: the model version, the index, the retrieval logs and the review dispositions are all available for inspection, which is rarely true of a hosted service.
Capable models,
inside the practice.
Privilege AI builds specialised models and private inference for work where confidentiality is the default.