Retrieval-augmented generation
Retrieval-augmented generation (RAG) is a pattern where a system first searches a collection of documents for passages relevant to a question, then gives those passages to a language model so its answer draws on them.
Also called: RAG
By Divya Sharma · Checked Nov 15, 2026 · How we review
A language model only knows what was in its training data, up to its knowledge cutoff. It cannot see your company wiki, last week's tickets or a contract signed yesterday. RAG closes that gap without retraining: you fetch the right text at question time and put it in the prompt.
How it works
Ahead of time, documents are split by chunking, turned into vectors by an embedding model and stored in a vector database. When a question arrives, the system embeds it, retrieves the closest chunks, often reorders them with a reranker, and passes the top few to the model with an instruction to answer from them.
The 2020 paper that named the idea joined a trained text generator with a searchable index of Wikipedia. Most RAG systems today skip that joint training and connect an off-the-shelf model to a separate retriever.
Example
A support bot gets "Can I change my plan mid-month?" It retrieves the billing policy paragraph and the plan FAQ, and the model answers from those, citing both.
Limits
RAG is only as good as its retrieval. If the right passage is never found, the model may fill the gap with a hallucination. It also struggles with questions about a whole corpus, such as "what are the main themes?", which is the case GraphRAG targets. The tools for each step are compared in the AI stack guide.
Related terms
- Vector database
- A vector database stores embeddings alongside their source records and finds the stored vectors closest to a query vector, usually with an approximate index so search stays fast across millions of items.
- Hallucination (AI)
- A hallucination is output from an AI model that sounds fluent and confident but is false, unsupported by its sources or made up, such as an invented citation, quote, statistic or API function.
- Embedding
- An embedding is a list of numbers (a vector) that represents a piece of text, an image or other data, produced so that items with similar meaning end up with vectors close together.
- Chunking
- Chunking is the step that splits long documents into smaller passages before they are embedded and indexed, so a retrieval system can find and return only the parts of a document that answer a query.
- Reranking
- Reranking is a second search step that takes a short list of candidate documents and reorders them by relevance to the query, using a more accurate but slower model than the first search.
- Agentic RAG
- Agentic RAG is retrieval-augmented generation run by an AI agent that decides when to search, what to search for and which source to use, and can search again if the first results fall short.