Input distribution.
Document length, source mix, language, format and volume. Moves first, because the world changes before the system does.
In production
nobody hands you
the answer.
A deployed model is measured without labels, late, and through proxies. Monitoring is the discipline of noticing degradation before a user reports it.
READ THE DEFINITION ↓DRIFT · PROXIES · SAMPLING
01 / DEFINITION
The short answer first.What is AI model monitoring?
AI model monitoring is the continuous measurement of a deployed model's inputs, outputs and downstream outcomes. Because correct answers are usually unavailable at the moment of use, monitoring relies on distribution comparisons, proxy signals and a sampled stream of human-reviewed items.
The defining constraint is the absence of ground truth. A pre-release evaluation knows the right answer for every item; a production system usually learns it much later, if at all. Monitoring therefore measures three things that are available — what is coming in, what is going out, and what happens next — and infers quality from changes in their relationship.
02 / WHAT TO WATCH
Four signals, in order of how early they move.Output distribution.
Score distribution, refusal rate, flag rate, answer length. A flag rate that halves overnight is a system event, not a quality improvement.
Retrieval behaviour.
Hit rate, empty-result rate and the share of answers with no supporting passage. Usually the first place a re-index breaks something.
Human disposition.
What reviewers do with the output. The slowest signal and the only one that directly measures usefulness.
The output distribution is the highest-value and most neglected of the four. It is cheap to compute, it has no labelling requirement, and most silent failures show up there first: a parser that stopped matching, a truncated context window, a prompt template that lost a section, a model endpoint that quietly changed version. None of these produce errors. They produce plausible output with a different shape.
03 / SILENT FAILURE
The failures that never raise an exception.| Failure | What it looks like | Detection |
|---|---|---|
| Empty retrieval | Fluent answers with no sources | Share of responses with zero retrieved passages |
| Context truncation | Later sections of long documents ignored | Answer quality by input length; token-count distribution |
| Parser drift | Correct answers scored as failures | Parse-failure rate, and a fixed golden set run hourly |
| Threshold staleness | Flag volume moves without a model change | Flag rate against a rolling baseline, per check |
| Upstream model change | Style and length shift overnight | Output length and vocabulary distribution; pinned version checks |
The last row is a specific argument for controlling the deployment. A hosted model that is updated by its provider changes behaviour underneath a system that has not changed at all, and the first indication is usually a shift in output shape. Where a model runs on infrastructure the organisation controls, the version changes when someone decides it does. That property is discussed in on-premise AI.
04 / ALERTING THAT SURVIVES
Designing thresholds people will not disable.Monitoring fails organisationally more often than technically. An alert that fires on ordinary variance is muted within weeks, and a muted alert is worse than none because it creates the impression of coverage.
- Alert on sustained movement, not single points. Compare a rolling window against a longer baseline, so one unusual batch does not page anyone.
- Alert per check, not per system. A single system-wide quality number is too noisy to threshold and too coarse to act on.
- Attach an action to every alert. If the only response is to look at a dashboard, the alert should be a report instead.
- Keep a sampled human review stream running continuously. A fixed number of items per week, reviewed against the same criteria as the evaluation set, is the only signal that keeps its meaning when everything else drifts.
That last practice is also what eventually produces live-work evidence. Benchmarks measure the model at a point in time; a sustained disposition record measures it in use, which is a different and stronger claim.
05 / QUESTIONS
Asked when an AI system goes into production.How do you monitor quality without labels?
+
Through three layers: distribution checks on inputs and outputs, which need no labels; proxy signals such as reviewer edit rates and escalations; and a small, continuous stream of human-reviewed items scored against the evaluation criteria. The first two detect change, the third measures quality.
What is model drift, exactly?
+
Two different things that get the same name. Data drift is a change in the input distribution; concept drift is a change in the relationship between inputs and correct answers. Data drift is detectable immediately and cheaply; concept drift usually is not, which is why a labelled review stream matters.
Do model weights drift?
+
Static weights do not change on their own. What changes is everything around them: the prompt, the retrieval index, the input mix, and — for hosted models — the version behind the endpoint. Attributing a quality change requires all of those to be pinned and recorded.
How much of the output should be human-reviewed?
+
Enough to detect a meaningful change within the time you are willing to be wrong. A fixed weekly sample sized against that tolerance is more useful than a percentage, because a percentage silently shrinks the review stream when volume drops.
Is monitoring different for agents?
+
Yes. Agent monitoring has to cover actions and tool calls, not just outputs, and the relevant unit is the trajectory rather than the response. See AI agent monitoring.
Notice it
before the user does.
Privilege AI builds monitoring for deployed AI systems: distribution checks, sampled review streams and per-check alerting.