Nothing relevant
was retrieved.
The most common and most fixable cause. Asked a question the sources cannot answer, a model trained to be helpful answers anyway. The fix is in retrieval and in an explicit refusal path.
Fluent,
and unsupported.
A hallucination is not a bug in the ordinary sense: the system is working as designed and the output is wrong. Detection means checking claims against sources, at scale.
READ THE DEFINITION ↓CLAIMS · SOURCES · SUPPORT
01 / DEFINITION
The short answer first.What is AI hallucination detection?
AI hallucination detection is the process of identifying statements in generated output that are not supported by the provided sources or by verifiable fact. In practice most systems measure groundedness — whether each claim traces to a retrieved passage — because attribution is checkable at scale and truth is not.
The distinction between groundedness and truth is the most useful thing to be clear about. A grounded claim is one supported by a cited source; a true claim corresponds to reality. They come apart in both directions: a model can faithfully repeat an error in a source document, and it can state something correct that appears nowhere in the retrieved material.
Most production detection targets groundedness, because it can be automated against a corpus the organisation controls. Truth checking requires an external authority and is generally limited to narrow domains with a canonical reference.
02 / WHY MODELS FABRICATE
Four mechanisms with different fixes.Nothing relevant
The shape is right,
the content invented.
Citations, case numbers, section references and dates follow strong surface patterns. A fabricated one looks exactly like a real one, which is why format validation catches nothing.
Sources disagree
and the model picks.
Given contradictory passages, a model typically synthesises a confident answer rather than reporting the conflict. Detecting disagreement between sources is a separate check.
A fourth mechanism is instruction pressure. A prompt that demands a specific output format — three bullet points, a named clause, a numbered citation — makes omission harder than invention. Where the honest answer is that the material does not contain the requested item, the format leaves no way to say so. Prompts that permit an explicit not-found result reduce fabrication measurably and cost nothing.
03 / DETECTION METHODS
What actually works, and what it costs.| Method | Catches | Cost | Limitation |
|---|---|---|---|
| Reference existence check | Citations, identifiers and sections that do not exist | Very low | Only works where a canonical index exists |
| Claim-to-source matching | Statements with no supporting retrieved passage | Low | Requires claim extraction, which is itself imperfect |
| Self-consistency sampling | Unstable answers that change between runs | Moderate | A confidently wrong model is consistently wrong |
| Judge-model review | Unsupported reasoning and overstated conclusions | Moderate | Inherits the judge's own blind spots |
| Domain expert review | Errors that require knowing the field | High | Does not scale; used to validate the layers above |
Reference existence checking is underused relative to its value. Where an organisation holds an authoritative index — of matters, contracts, filings, products or clauses — verifying that every cited identifier resolves is close to free and eliminates the most damaging class of fabrication outright. It should be the first check built, not the last.
Self-consistency deserves a caveat. Sampling the same prompt several times and comparing answers detects instability, which correlates with fabrication but is not the same thing. It is a useful prioritisation signal and a poor gate.
04 / MEASURING THE RATE
Turning detection into a number.Detection is only useful
if the miss rate is known.
Every detector has a false-negative rate. Reporting flagged items without estimating what was missed overstates the protection.
The practical method is a labelled subset: take a sample of output, have a domain expert mark every unsupported claim, and score the detector against that. This gives the two numbers that matter — what share of fabrications are caught, and what share of flags are spurious — and it is the only way to know whether the detector is contributing.
Those numbers then set the authority the detector may hold, which is the same logic applied to supervision generally: a check with strong ranking quality can gate, one with weak ranking quality can only report. Privilege AI's supervision work applies exactly this structure, with one of 45 checks refused outright because its ranking quality was insufficient. See AI model supervision and the MKC2 record.
05 / QUESTIONS
Asked when a fabricated fact would be expensive.Can hallucinations be eliminated?
+
Not by any current method. They can be made rarer through better retrieval, reduced in impact through an explicit refusal path, and made visible through attribution checking and review. A system design that assumes elimination is the actual risk.
Does retrieval-augmented generation solve hallucination?
+
It reduces one cause and introduces another. Grounding answers in retrieved passages helps when retrieval succeeds; when it returns nothing relevant, the model may still answer, now with the appearance of sourcing. Measure the empty-retrieval rate as a first-class signal. See RAG evaluation.
Are fabricated citations detectable automatically?
+
Where a canonical index exists, yes, and reliably: the identifier either resolves or it does not. The harder case is a real citation attached to a proposition it does not support, which requires reading the source and comparing it to the claim.
Is a low hallucination rate on a benchmark meaningful?
+
Only for material like the benchmark. Fabrication rates are strongly domain-dependent, and a model that is reliable on general questions can be markedly worse on specialist vocabulary where its training data is thin. Measure on your own corpus.
Who should review flagged claims?
+
Someone able to tell an unsupported claim from an unfamiliar one, which usually means a domain expert rather than a generalist reviewer. The queue should be ordered by calibrated confidence so that expert time goes to the highest-risk items first.
Check the claim
against the source.
Privilege AI builds retrieval and supervision systems that measure groundedness and report what the detector misses.