Law 12 · Retrieval & Memory
Grounding Is Not a Guarantee
Retrieval reduces hallucination; it does not eliminate it.
The principle
Vendors marketed RAG legal tools as 'hallucination-free', yet a Stanford audit found they still hallucinated 17–33% of the time. Handing the model a source doesn't force it to use that source faithfully — it can misread, over-generalize, or cite a real document for a claim the document never makes. Grounding lowers the floor on errors; it never reaches zero.
Why it happens
Placing a source document in context biases the model toward it but does not bind generation to it, because decoding still samples from a distribution shaped by parametric priors, paraphrase pressure, and the instruction to be helpful and complete. The model can faithfully retrieve a real passage and still attach a claim the passage never makes, over-generalize a narrow statement, or stitch two spans into an unsupported synthesis. Dedicated grounding benchmarks exist precisely because this gap is measurable: Google DeepMind's FACTS Grounding evaluates whether long-form answers are fully supported by a provided document and disqualifies responses that introduce any unsupported claim, and even strong models leave a visible non-grounded fraction. The lesson is that grounding lowers the error floor but never reaches zero, so faithfulness must be verified per claim rather than assumed from the presence of a source.
Watch for
- A grounded system is described to stakeholders as hallucination-free or hallucination-proof.
- No step checks that each generated claim is actually entailed by a retrieved span.
- Citations are attached to answers but nobody has verified the cited passage supports the specific claim.
In practice
Your team ships a contracts assistant, tells the client it is 'hallucination-free because it uses RAG', and a month later it cites a real clause for an indemnity term that clause never mentions. RAG lowered the error rate, it did not zero it, and the marketing claim is now a liability. Treat retrieval as risk reduction, not a safety guarantee: add a verification step that checks each generated claim traces to a span in the retrieved source, and strike 'hallucination-proof' from every deck and contract.
Apply it
- Add a verification pass that checks each output claim is entailed by a specific retrieved span before returning it.
- Require inline attribution at the claim level so faithfulness can be audited rather than trusted.
- Frame retrieval as risk reduction in all messaging and remove absolute safety language from decks and contracts.
The takeaway
Treat 'we use RAG' as risk reduction, not a safety claim. Verify that generated claims actually trace to the retrieved passage, and never advertise grounded systems as hallucination-proof.