For Heads-of · Practitioner
Retrieval-Augmented Generation (RAG)
A technique in which a language model retrieves relevant documents or data before generating a response, grounding its output in source material rather than in its training data alone.
- RAG
- retrieval augmented generation
- grounding
- fact checking
How it works
The system retrieves documents similar in meaning to the query, usually by comparing embeddings, and passes them to the language model as context. The model then answers using the retrieved material.
Benefits
RAG reduces confabulation by grounding answers in sources, lets a model use private or recent information that was not in its training data, and allows updates without retraining. It reduces confabulation; it does not eliminate it.
Governance advantages and new risks
The system can cite which documents informed an answer, which supports accountability and review, although a citation shows what was retrieved, not that the model used it faithfully. Retrieval also adds risks of its own: poisoned or over-shared documents, and the vector and embedding weaknesses that vector store and embedding access controls address.
Related Terms
Context Window
The maximum length of text (measured in tokens) that a language model can consider at once when generating a response.
Data Poisoning
An attack in which an adversary deliberately corrupts data an AI model learns from, to degrade it or plant hidden behaviour. It now reaches beyond training data to fine-tuning sets, retrieval corpora and agent memory, and a poisoned model can pass ordinary evaluation.
Embedding
A numerical representation of text, images or other data in a high-dimensional space, where items with similar meaning sit close together.
Hallucination
Output that is fluent, confident and false. It arises from the same mechanism that produces correct output (a model predicting plausible continuations), which is why it is best understood as a property of the technology to be bounded rather than a defect to be fixed.