Golden cases.
A small, hand-curated set of inputs with known-good outputs. Fast, deterministic, and the first thing to run. If a golden case breaks, stop.
MKC2 results ↗AI model testing turns a one-off evaluation into a suite that runs on every change, pins every version and refuses a release that moves the wrong way.
01 / DEFINITION
The short answer first.AI model testing is the repeatable execution of a fixed set of cases against a model or pipeline, with pinned versions and recorded results, so that any change in behaviour can be attributed to a specific change in the system. Testing answers whether a build regressed; evaluation answers how good it is.
The distinction matters operationally. An evaluation is an event with a report. A test suite is infrastructure that runs unattended and produces a pass or fail. Teams that only do the former discover regressions from users; teams that only do the latter often cannot say what their absolute quality level is, only that it has not dropped.
Both are needed, and they usually share a data set but not a workflow. The evaluation set is large, expensive and read carefully by a person. The test set is small, fast, deterministic where possible, and read by a pipeline.
WORKED EXAMPLE
Imagine a model that converts purchase requests into records accepted by a purchasing application. A request for twelve packs of four filters must preserve the distinction between pack quantity and individual units. A response containing forty-eight in the wrong field can pass a basic JSON check while creating an incorrect order. Test the consumer of the output as well as its syntax.
Save the de-identified request, the schema version and the expected fields. Add a nearby case that asks for twelve individual filters. Feed both outputs to the real parser in a test environment without submitting an order. Check that the application displays the intended unit, quantity and explanation of any assumption.
Retain a malformed case with no pack size. The correct behaviour may be to ask for clarification. If the new model guesses four because the previous example used four, record that as a separate failure. A useful release test can show the exact application state a regression would have produced and the condition that keeps it out of production.
02 / SUITE DESIGN
A small, hand-curated set of inputs with known-good outputs. Fast, deterministic, and the first thing to run. If a golden case breaks, stop.
Every input that has ever produced a defect, kept forever. This is the layer that stops a fixed bug from returning two releases later.
Inputs designed to break the system: injected instructions, contradictory sources, truncated documents, wrong language, empty retrieval results.
A sampled run over the large evaluation set, reported with confidence intervals rather than treated as a binary gate.
Layers one to three are binary gates. Layer four is not, and trying to make it one is a common source of flaky pipelines: a two-point move on a sampled metric is usually noise, and a gate that fires on noise gets disabled within a month. Gate on the deterministic layers, alert on the statistical one.
The adversarial layer is the one most teams add last and need first. For systems that read untrusted documents, it should include the prompt-injection cases described in AI agent prompt injection, because a retrieval pipeline that ingests a hostile PDF is an injection target whether or not it has agentic tools.
03 / PINNING
A test result is only attributable if everything that can change is recorded. In a language model pipeline that is at least six things: the model weights, the decoding parameters, the system prompt, the retrieval index, the output parser and the scoring function. A quality drop caused by a re-indexed corpus looks identical to one caused by a model swap unless both are pinned.
Two of these are routinely forgotten. The output parser silently determines the score whenever a model changes its formatting habits, and a parser that fails to extract an answer usually records a wrong answer rather than an error. The scoring function, particularly a judge model, has its own version and its own drift; pinning the judge is as important as pinning the model being judged.
04 / GATES
An absolute threshold ages badly and gets waived. A gate on movement relative to the current production build survives, because it always compares like with like.
A model that fails a check should lose the authority attached to that check, not the whole deployment. Per-check authority makes partial failure survivable.
Overrides happen. An override with a name, a date and an expiry is governance; an override applied silently in a pipeline configuration is a future incident.
Per-check authority is how MKC2 handles partial failure: of 45 checks, 25 carry report-only authority and one is refused outright for insufficient ranking quality. The model still ships; the check does not. That structure means a weak check degrades one flag rather than blocking the whole system, and it is the practical form of the argument made in AI model supervision.
05 / QUESTIONS
Pin the decoding parameters to greedy for the deterministic layers so they behave like ordinary unit tests, and accept variance only in the sampled statistical layer. Where the production system samples, run the sampled layer several times per item and compare distributions rather than single values.
Small enough to run in minutes and be reviewed by a person in an afternoon, which in practice means tens of cases, not thousands. Its job is to fail loudly on obvious breakage. The large set does the measuring.
The cases come from whoever understands the failure cost, which is usually the domain expert rather than the ML engineer. The infrastructure belongs to engineering. Splitting it that way keeps the suite anchored to real consequences instead of drifting towards whatever is convenient to measure.
The same thing that happens when any production check fails: the affected capability is reduced to report-only or switched off while the cause is found. That is only possible if authority is attached to individual checks rather than to the deployment as a whole.
No. Testing establishes that the system behaves as it did when a human last judged it acceptable. It does not extend that judgement to new categories of input. See human-in-the-loop AI.
WORK WITH PRIVILEGE AI
Results you can repeat.
We build test setups that record every version, so a result can be reproduced later.
Catch regressions early.
A standing set of test cases shows when an update makes things worse.
Safer releases.
Clear checks before each release give your team confidence to ship.
Talk with Privilege AI
Tell us which user action or output contract must keep working after your next model release.
CONTACT PRIVILEGE AI
Tell us which user action or output contract must keep working after your next model release.
Privilege AI builds evaluation harnesses that pin every version, keep a regression corpus and report per-check rather than per-model.