Answers correct
Against 83% for the stock model,
scored on the same material.
PRIVILEGE AI / RESEARCH & ENGINEERING
Intelligence,
under scrutiny.
We develop specialised models and study how they reason,
retain knowledge, and evaluate the work of other models.
SCROLL TO EXPLORE ↓MODELS · KNOWLEDGE · SYSTEMS
01 / MODEL RESEARCH
Developed, fine-tuned and evaluated in-house.AI that questions
AI.
MKC2 is a specialised supervision model developed by Privilege AI: a model that evaluates AI-generated work and the actions of AI agents. It runs entirely offline.
MKC2 is the first fully offline, fine-tuned supervision model built specifically to judge legal drafting and legal-agent actions. It ranks defective work above sound work at 0.962, against 0.728 for the base model it was fine-tuned from.
Explore the measured results ↓MEASURED. NOT ASSUMED.
It finds the flawed work.
96/100
Put one flawed passage next to one sound passage. MKC2 picks out the flawed one 96 times out of 100.
The stock model it was built from manages 73. The model it replaces, 65.
Checks we refuse to rely on
One check could not be shown to work,
so it is switched off, not shipped.
Unseen items tested
None of them seen during training.
45 separate checks on each.
Calibration record, 21 September 2026. MKC2 is fine-tuned from Qwen3-8B. Every figure is measured on 4,403 held-out items, with the model it replaces re-scored in the same run on the same material. No attorney dispositions have been recorded, so none of this is evidence that MKC2 agrees with a lawyer on live work. MKC2 reports; it does not act.
Results in context +
MKC2 leads on all four measures against both the model it replaces and the stock base model: ranking quality 0.9623 against 0.6457 and 0.7281; Brier score 0.0620 against 0.1409 and 0.1274; calibration error 0.0610 against 0.1785 and 0.1363; correct 93.17% against 80.73% and 83.09%.
The gain is largest on the measure that decides whether a confidence figure is usable at all: how well the model ranks defective work above sound work. The model it replaces scored 0.6457 there, below the stock base model.
Of 45 questions, one is refused outright for insufficient ranking quality and 25 carry report-only authority, changing nothing. Questions that cannot be shown to work are not relied upon.
MKC2 runs in review-only mode: it reports and flags, it does not block or act on its own. That boundary is deliberate. These are benchmark results on held-out data, not live-matter outcomes.
02 / RESEARCH DIRECTIONS
Questions we are working on.MODEL SUPERVISION
When should a model
trust another model?
We study the reliability of AI-generated work and the role of specialised models in reviewing it. Our work in HIGHCOURT brings these questions into a demanding application.
Read the evaluation note ↗EVALUATION & AUTHORITY
Capability is measured.
Authority is earned.
Strong benchmark performance is one part of responsible deployment. We distinguish measured capability from the responsibilities a model should be given.
The reported model operates in review-only mode.
03 / RESEARCH & ENGINEERING
The work spans the full system.Models.
Knowledge.
Infrastructure.
We work across model adaptation, retrieval and private deployment. Each layer changes what a system can do, what it can know, and how its behaviour can be evaluated.
Training and
fine-tuning.
Dataset construction, model training and fine-tuning, with task-specific evaluation and held-out comparisons.
Retrieval and
knowledge.
Retrieval-augmented generation and knowledge systems. Research into the relationship between context and model performance.
Inference and
systems.
Local inference, application integration and on-premise infrastructure. Security boundaries considered throughout the system.
04 / PRIVATE SYSTEMS
Control is part of the architecture.Private inference.
Explicit boundaries.
Deployment is part of the research problem. Data access, model authority and operational constraints shape how AI behaves outside a benchmark.
On-premise inference.
Local model execution with explicit hardware, network and operational constraints.
Access with intention.
Permissions and retrieval boundaries define which information a model can access and which actions it can propose.
Security from the start.
Sensitive data, model behaviour and deployment risk are considered throughout development and evaluation.
CONTROLLED ENVIRONMENT
Data — Models — Applications
ONE CONTROLLED BOUNDARY05 / ABOUT PRIVILEGE AI
Experience compounds.OUR FOUNDATION
Built on decades
of experience.
A continuous practice
of engineering.
Built on decades of experience in software engineering and data science, Privilege AI brings that foundation to the development of specialised models and AI systems.
We connect that experience to the whole system: how data is structured, how software is built, and how intelligence becomes useful.
Web developmentTechnical SEOData scienceAI engineering
CONTACT
Start a
conversation.
For research, technical enquiries and collaborations.
Tell us a little about what you’re working on.