Why it happens, and why that framing matters

A language model generates text by predicting likely continuations from patterns in training data. It has no separate faculty that checks assertions against the world, and no internal representation of the difference between a fact it encountered and a plausible-sounding construction. Fluency and accuracy are produced by the same process, which is why confidence in the output tells you nothing about its truth.

The governance consequence follows directly. A defect can be fixed and a property must be bounded. An organisation that treats hallucination as a bug awaiting a vendor patch will keep deferring the control work; one that treats it as a property will build verification, grounding and human review proportionate to the consequence. The second is the only posture that survives contact with a regulator.

The word is contested, and the alternatives are in your frameworks

"Hallucination" is criticised for anthropomorphising — a model is not perceiving something that is not there, it is generating a statistically plausible sequence. Two frameworks the site already maps against use different terms, and knowing which is which is practically useful:

  • NIST AI 600-1, the Generative AI Profile, uses confabulation, which is the more accurate borrowing: confident narrative generated to fill a gap, without awareness that it is doing so.
  • The OWASP GenAI LLM Top 10 files it under Misinformation, at LLM07 in the 2026 list — a framing that centres the downstream harm rather than the mechanism.

The site's register entry is misinformation and confabulation, which spans both. Use whichever term your audience does; know that a framework citation may use another.

Where it shows up

Fabricated citations and non-existent sources; invented statistics and figures; plausible but incorrect procedural or legal detail; and confident answers about entities that do not exist. Frequency rises with topics thinly represented in training data, with highly specific factual queries, and — importantly — with questions whose premise is false, where the model will often elaborate on the premise rather than challenge it.

The last of these is the most dangerous in a business context, because the user has already supplied the error and receives it back with corroboration.

Why the risk is about the workflow, not the rate

Two systems with identical hallucination rates can carry entirely different risk, because what matters is what happens to the output next. A drafting assistant whose output a subject-matter expert reviews is a manageable exposure. The same model publishing to customers, or feeding a downstream automated step, is not.

This is why measuring a hallucination rate in isolation is close to useless as a governance activity, and why the useful question is: where does an unverified assertion from this system reach a consequential destination?

What actually reduces the consequence

Grounding. Retrieval-augmented generation, with citations the user can follow, so an assertion can be checked against a source rather than trusted. It reduces the rate substantially and does not eliminate it — a model can misread or over-extend a retrieved source.

Verification for consequential claims. Factuality and hallucination verification is the control, and it should be targeted: verifying everything is unaffordable, verifying the categories that matter is not.

Structural output validation. Where output has a checkable shape — a citation that should resolve, a figure that should reconcile, an identifier that should exist — validate it deterministically. See output validation guardrails.

Surfacing uncertainty rather than suppressing it. A system that can say it does not know is more useful than one that cannot, and product pressure runs the other way.

Human review where the consequence warrants it. Human-in-the-loop review remains the most reliable control available, and the test is whether the reviewer has the time and the standing to disagree.