Ungrounded or misattributed answers

What it is

Retrieval is meant to keep a model honest: it answers from documents and shows which ones. The promise fails in several ways. The model may draw on its own memory and attach a retrieved document as decoration. It may cite the right document for a claim the document does not make. It may blend two sources into a statement neither contains. The user sees a citation and reasonably assumes the claim has been checked. A wrong answer with a reference is trusted more than a wrong answer without one. The general problem is covered in misinformation and confabulation.

How it happens

• Retrieval returns nothing relevant and the model answers regardless.
• The model is not constrained to the retrieved passages.
• Citations are attached at document level, so the supporting passage cannot be checked.
• Chunks are cut in a way that separates a statement from its qualification.

Why it matters

The organisation has told users the answer is sourced. Decisions are made on that footing, and the source is rarely opened. In regulated advice or legal work, a misattributed authority is as damaging as an invented one.

Warning signs

• Nobody has measured how often citations support the claims they accompany.
• The system gives a confident answer when retrieval found nothing.
• Citations point to whole documents, not passages.
• Users report that the linked source does not say what was claimed.