Agentic RAG
Agentic RAG is retrieval-augmented generation run by an AI agent that decides when to search, what to search for and which source to use, and can search again if the first results fall short.
Also called: agentic retrieval, agentic retrieval-augmented generation
By Divya Sharma · Checked Nov 15, 2026 · How we review
Classic retrieval-augmented generation is a fixed pipeline: embed the question, retrieve top chunks, generate once. That fails when a question needs several steps, such as comparing two contracts or tracing a bug through logs and code. Agentic RAG hands retrieval to an AI agent running in a loop.
How it works
The model gets search as one or more tools through function calling: a vector index, a keyword index, a SQL database, the web. In each turn of the agent loop it decides whether it needs more information, writes a query (often a rewritten one), reads the results and either searches again or answers. It can split a question into sub-questions and route each to a different source.
Example
Asked "Did churn rise last quarter in the regions where we raised prices?", the agent queries the pricing change log, then the churn table by region, then compares them before answering.
Early forms
Self-RAG (2023) trained a model to decide for itself when to retrieve and to critique the passages it got. Current systems mostly prompt general models to do this with tools rather than training a special model.
Trade-offs
Agentic RAG handles harder questions, but each extra search adds model calls, latency and cost, and the agent can loop or stop too early. Cap the number of steps, log each tool call with LLM observability tools, and keep plain RAG for simple lookups. Deep research products are a long-running form of this pattern.
Related terms
- Retrieval-augmented generation
- Retrieval-augmented generation (RAG) is a pattern where a system first searches a collection of documents for passages relevant to a question, then gives those passages to a language model so its answer draws on them.
- AI agent
- An AI agent is a system in which a language model decides its own next steps: it calls tools, reads the results and repeats until the task is done, instead of following a fixed script.
- Agent loop
- An agent loop is the repeating cycle at the heart of an AI agent: the model asks for a tool call, the application runs it and returns the result, and the model decides what to do next.
- Query rewriting
- Query rewriting changes a user's question before search, for example by clarifying it, splitting it or adding related terms, so the retriever is more likely to find the passages that answer it.
- Tool calling
- Tool calling, also called function calling, is a model API feature where you describe functions with a schema, the model replies with a structured request to call one, and your code runs it and returns the result.
- Deep research (AI)
- Deep research is an AI agent feature that plans a research question, runs many web searches, reads the sources it finds and writes a long report with citations, taking minutes rather than seconds.