AI Glossary
RAG (Retrieval-Augmented Generation)
Also known as: Retrieval-augmented generation
RAG is the technique of retrieving relevant passages from a document base and handing them to the LLM along with the question, so the answer is grounded in real, current sources.
Why use RAG
An LLM doesn’t know your company’s documents or anything after its training cutoff. Instead of training a new model, RAG retrieves what’s relevant at question time and places it in the prompt.
The three steps
- Index: split documents into chunks and turn each chunk into an embedding.
- Retrieve: embed the question and find the closest chunks.
- Generate: send the question plus chunks to the LLM, instructing it to answer only from them and cite the source.
What usually goes wrong
Chunks that are too large, retrieval that pulls irrelevant content and no evaluation at all. A production RAG system needs retrieval-quality metrics, not just answer metrics.
Go deeper in the book
AI for SREs
Your LLM returned HTTP 200. And it was wrong. Learn to run AI with SRE discipline.