Taken on its own
MKC2 reports and flags.
It does not block or act.
MKC2 results ↗MKC2 is a specialised supervision model developed by Privilege AI. It evaluates AI-generated work and the actions of AI agents, and it runs entirely offline.
01 / THE MODEL
What MKC2 is.MKC2 is a supervision model developed and fine-tuned by Privilege AI. It reviews AI-generated legal drafting and the actions of legal AI agents, ranking defective work above sound work so that scarce expert attention goes where the risk is. It runs entirely offline, in review-only mode.
MKC2 was built for HIGHCOURT, where the review problem is concrete: more AI-assisted drafting is produced than can be read line by line, and the cost of a defect reaching a client is high. It is, as far as Privilege AI is aware, the first fully offline, fine-tuned supervision model built specifically to judge legal drafting and legal-agent actions.
Two design decisions define it. It is specialised: the judgement standard for what counts as defective legal work is long, expert and stable, which is the shape of problem fine-tuning suits. And it is bounded: it reports and flags, and it does not block or act on its own.
WORKED EXAMPLE
Select one judgement from the published calibration record and identify its intended use. Is it a report for an attorney to inspect, or a signal someone might be tempted to use to block work? Keep the measured comparison attached to the authority granted. A strong ability to rank a defective passage above a sound one does not establish agreement with an attorney on a live matter.
For an illustrative comparison, give a reviewer two drafting passages that differ in their treatment of a source condition. Ask what MKC2 flagged, which words support the flag and which decision remains with the attorney. Include a case where the evidence does not establish a defect. A useful supervision report allows uncertainty to remain visible and gives the reviewer enough context to reject an unsupported warning.
When considering a different workflow, identify the new task, source material and human reviewer first. Build a held-out comparison for that setting rather than carrying a benchmark score or grant of authority across applications without a new test.
02 / THE RECORD
Measured, not assumed.Calibration record, 21 September 2026. Every figure is measured on held-out items, with the model being replaced re-scored in the same run on the same material.
Put one flawed passage next to one sound passage. MKC2 picks out the flawed one 96 times out of 100.
The stock model it was built from manages 73. The model it replaces, 65.
MKC2 reports and flags.
It does not block or act.
One check could not be shown to work,
so it is switched off, not shipped.
None of them seen during training.
45 separate checks on each.
Calibration record, 21 September 2026. Every figure is measured on held-out items, with the model it replaces re-scored in the same run on the same material. No attorney dispositions have been recorded, so none of this is evidence that MKC2 agrees with a lawyer on live work. MKC2 reports; it does not act.
MKC2 leads on all four measures against both the model it replaces and the stock base model, including how far its stated confidence can be trusted.
The gain is largest on the measure that decides whether a confidence figure is usable at all: how well the model ranks defective work above sound work. The model it replaces scored 0.6457 there, below the stock base model.
Of 45 questions, one is refused outright for insufficient ranking quality and 25 carry report-only authority, changing nothing. Questions that cannot be shown to work are not relied upon.
MKC2 runs in review-only mode: it reports and flags, it does not block or act on its own. That boundary is deliberate. These are benchmark results on held-out data, not live-matter outcomes.
03 / HOW TO READ IT
| Measure | MKC2 | Stock base | Model replaced | What it licenses |
|---|---|---|---|---|
| Ranking quality | 0.9623 | 0.7281 | 0.6457 | That a threshold on the score can separate defective from sound work |
| Answers correct | Best | Second | Third | That the model is competent at the checks it is given |
| Brier score | Best | Second | Third | That predictions are both right and appropriately confident |
| Calibration error | Best | Second | Third | That a stated confidence can be shown to a reviewer |
The ranking figure is the one to read first. A score with weak ranking quality cannot support any routing rule, however accurate the model is on average — there is no threshold that separates the classes. The model being replaced scored 0.6457 there, below the stock base model, which is the clearest statement of why the work was done.
Definitions for each measure are set out in LLM evaluation metrics, and the reason calibration is reported separately from accuracy is developed in AI model calibration.
04 / THE BOUNDARY
MKC2 runs in review-only mode. It reports and flags. It does not block work, change a document or take an action on its own.
No attorney dispositions have been recorded. The record measures behaviour on held-out evaluation data, which is a different and weaker claim than agreement on live matters.
One check of 45 is refused outright for insufficient ranking quality, and 25 carry report-only authority. A check that cannot be shown to work is switched off, not averaged in.
These boundaries are part of the design rather than caveats attached to it. The reasoning is set out in AI model supervision: capability is measured, and authority is granted separately, per check, by the organisation carrying the risk.
05 / DEPLOYMENT
MKC2 runs entirely offline. Reviewed material, which in a legal setting is privileged by default, never leaves the environment it is held in.
Fine-tuned from an established base model rather than built from scratch, which keeps the hardware requirement modest and the inference cost predictable.
Each of the 45 checks carries the authority its own measured evidence supports, from refused through report-only to flagging for review.
Deployment considerations for models of this class — hardware, quantisation, throughput and the network boundary — are covered in on-premise LLM deployment and private AI inference. The legal-specific case is set out in private legal AI.
06 / QUESTIONS
It reads AI-generated work and proposed AI agent actions, applies 45 separate checks to each item, and reports findings with a confidence score. The output is a prioritised set of flags for a human reviewer, not a decision. It does not block or change anything.
Placed against one sound passage, MKC2 scores a defective passage higher about 96 times in 100. That ordering property is what makes a review queue useful: the items most likely to be defective rise to the top. The stock base model it was fine-tuned from scores 0.7281 on the same items.
That has not been measured. No attorney dispositions have been recorded, so the record shows performance on held-out evaluation data and nothing about agreement on live matters. Establishing that would require a sustained disposition record from real review work.
Yes. It runs entirely offline, which is the reason it can be used on privileged material. No prompt, document or output is sent to an external inference service.
Because their measured evidence supports reporting a finding but not gating on it. Authority is granted per check, by the strength of that check's ranking quality and calibration, rather than by the model's overall score. One further check is refused outright and switched off.
MKC2 was developed for HIGHCOURT. For enquiries about supervision models, evaluation work or related research, the fastest route is a direct conversation with the team through the contact form.
WORK WITH PRIVILEGE AI
We can help define a review rubric, compare a candidate with its base model on held-out examples, and identify which findings need human review. For MKC2, the starting point is the published calibration record and its review-only operating boundary.
Talk with Privilege AI
Tell us which review question you need to evaluate and what a human decision-maker would need to see in a supervision report.
CONTACT PRIVILEGE AI
Tell us which review question you need to evaluate and what a human decision-maker would need to see in a supervision report.
The calibration record above summarises what has been measured. For the methodology behind it, or for supervision work of your own, start a conversation.