Fine-tuning.
Training on a narrow task sharpens the output distribution. The model becomes more accurate and more confident, and the second usually moves further than the first.
Confidence is
a claim too.
A model that says it is 90% sure should be right about nine times in ten. Calibration is the measurement of that promise, and it decides what can be automated.
WHAT CALIBRATION MEANS ↓BRIER · ECE · THRESHOLDS
01 / DEFINITION
The short answer first.What is AI model calibration?
AI model calibration is the correspondence between a model's stated confidence and its observed accuracy. A calibrated model that reports 0.8 confidence is correct about 80% of the time at that level. Calibration is measured separately from accuracy, because a highly accurate model can still be badly calibrated.
Calibration is the property that turns a model score into an operational control. Every automation rule of the form act when confidence is above X, route to a human below it assumes the score means something consistent. If it does not, the rule silently sends the wrong items to the wrong place, and the failure is invisible because the model still looks accurate on average.
02 / WHY IT DEGRADES
Four ways a good model becomes overconfident.Instruction tuning and RLHF.
Preference training rewards decisive, fluent answers. Hedging is penalised by raters, so the model learns to sound certain regardless of evidence.
Distribution shift.
Calibration is fitted on one input distribution. New document types or a new intake process break it without changing the weights at all.
Verbalised confidence.
Asking a model to state a percentage in text produces numbers clustered on round values and only loosely related to its actual token probabilities.
The fourth point is worth stating plainly because it is now so common. A self-reported confidence in prose is a generated string, not a measurement, and it should be validated against outcomes before any threshold is built on it. Sometimes it turns out to be informative. It is never safe to assume so.
03 / MEASURED EVIDENCE
What a calibration record looks like.Two numbers,
reported separately.
Privilege AI reports calibration for MKC2 with two figures: a Brier score, which combines correctness and confidence, and an expected calibration error, which isolates the confidence gap. Both are measured on 4,403 held-out items alongside the model being replaced and the stock base model, scored in the same run.
Brier score
Stock base model 0.1274,
model it replaces 0.1409. Lower is better.
Calibration error
Stock base model 0.1363,
model it replaces 0.1785. Lower is better.
Report-only checks
Checks that carry no authority to change
an outcome, by design.
Calibration record, 21 September 2026. Figures are measured on held-out evaluation data, not on live matters. MKC2 runs in review-only mode: it reports and flags, it does not act.
The structural point in that record is the third figure. Where a check's ranking quality or calibration is not strong enough to support a decision, the check is given report-only authority or switched off entirely rather than being averaged into a headline number. Calibration is not only a property to measure; it is the input to how much authority a model is granted, which is the argument developed in AI model supervision.
04 / RECALIBRATION
Fixing confidence without retraining.Recalibration is a post-processing step applied to a trained model's scores. It does not change what the model knows; it changes what its numbers claim.
- Temperature scaling. One parameter fitted on a validation set, applied to the logits. Cheap, hard to overfit and usually the first thing to try.
- Isotonic regression. A non-parametric monotonic fit. More flexible, and it needs considerably more validation data to avoid overfitting the calibration curve itself.
- Per-slice calibration. Fitting separately for document types or question types that behave differently. Often the largest single improvement, and the one most often skipped.
Two constraints are easy to get wrong. Recalibration must be fitted on data that is neither the training set nor the final held-out set, or the reported calibration error is optimistic. And because recalibration is monotonic, it improves calibration without improving ranking quality: a model that cannot order defective work above sound work will not be rescued by scaling its scores.
05 / QUESTIONS
Asked when a confidence threshold is being designed.Can a model be accurate but badly calibrated?
+
Routinely. Accuracy counts how often the top answer is right; calibration asks whether the confidence attached to it is honest. A model that is right 93% of the time but says it is 99% sure on everything is accurate and unusable for routing, because nothing can be separated by its score.
Does calibration matter if a human reviews everything?
+
Yes, because the confidence score determines review order and reviewer attention. An overconfident flag gets read as more serious than it is, and a reviewer who learns that high-confidence flags are often wrong starts discounting all of them. Calibration is what keeps a queue triageable.
How much validation data does recalibration need?
+
Temperature scaling can work with a few hundred labelled items. Isotonic regression and per-slice calibration need enough items in every bucket or slice to estimate a rate, which in practice means thousands. Under-powered per-slice calibration is worse than none, because it fits noise.
Is expected calibration error sensitive to how it is computed?
+
Yes. The number depends on the number of buckets and whether buckets are equal-width or equal-count. Different choices can move it noticeably, so the bucketing scheme should be published with the figure and kept fixed across comparisons.
How often should calibration be re-checked?
+
Whenever the input distribution can have moved — a new client, a new document source, a new intake process — and on a fixed schedule regardless. Calibration degrades under distribution shift without any change to the model, which is why it belongs in AI model monitoring rather than only in pre-release testing.
Find out what
the score is worth.
Privilege AI measures ranking quality and calibration separately, and reports the checks that cannot carry a decision.