RAG, grounding and hallucination control
Retrieval-augmented generation fetches relevant passages at query time and puts them in the prompt, so the model answers from sources instead of memory. The standard fix for private, missing or out-of-date knowledge.
By the end
Build a retrieval pipeline outline, write grounding instructions that allow an honest I don't know, and choose between prompting, RAG and fine-tuning for a given problem.
Retrieval-augmented generation (RAG) fetches relevant documents at query time and puts them in the prompt, so the model answers from sources instead of from memory. It is the standard fix for missing, private or out-of-date knowledge.
Learn
- Pipeline. Split documents into chunks, embed them, store them in a vector database, retrieve the top matches for each query, then answer from those passages.
- Grounding instructions. Answer only from the provided context, say "I don't know" when the answer is not there, and quote or cite the passage used.
- Other hallucination controls. Lower the temperature for factual tasks, ask for citations, and run a second pass that checks claims against the sources.
- Where RAG fails. Bad chunking, retrieval that misses the right passage, too many passages, or sources that contradict each other.
Prompting vs RAG vs fine-tuning
| Approach | Changes | Reach for it when |
|---|---|---|
| Prompting | Instructions and examples per request | Behaviour, tone or format needs adjusting |
| RAG | What the model can see at answer time | Facts are private, large or change often |
| Fine-tuning | The model's weights | You need a consistent style or narrow skill at scale; poor for adding facts |
Drill
- Build a small RAG over 10 documents with any vector store (pgvector works).
- Ask 10 questions, 3 of which the documents cannot answer.
- Check that the model says it does not know on those 3, and tighten the prompt until it does.
Check yourself
- Why is fine-tuning a weak way to teach a model new facts?
- Name two reasons a RAG system can still hallucinate.
Test yourself
6 multiple-choice questions and 8 flashcards on this lesson, shuffled every time, with the reason behind every answer.