Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Chunking

Chunking is the step that splits long documents into smaller passages before they are embedded and indexed, so a retrieval system can find and return only the parts of a document that answer a query.

Also called: document chunking, chunk size, chunk overlap, semantic chunking

By Divya Sharma · Checked Nov 15, 2026 · How we review

A language model cannot read a whole document library at once, and an embedding of a 200-page PDF says little about any single page. Chunking cuts the source into passages small enough to embed, search and paste into a prompt. It is the first design choice in a retrieval-augmented generation pipeline, and a bad choice here limits everything downstream.

How it works

The simplest method uses a fixed size: every N words, characters or tokens. The original RAG paper split each Wikipedia article into 100-word chunks. Most splitters add chunk overlap, repeating a few sentences at each boundary so an idea that spans two chunks is not lost. Recursive splitters try natural breaks first (sections, then paragraphs, then sentences) and fall back to smaller units only when a piece is still too long. Structure-aware chunking uses the headings and tables found during document parsing.

Semantic chunking

Semantic chunking places breaks by meaning instead of length. It embeds each sentence and starts a new chunk where the similarity between neighbouring sentences drops, so each chunk holds sentences about one topic. It costs an embedding call per sentence, and the breakpoint threshold needs tuning per corpus.

Trade-offs

Small chunks match queries precisely but lose context; large chunks keep context but dilute the embedding and use more of the context window. There is no universal best size, so test a few settings against real questions. The chunking stage guide compares the tools.

Related terms

Retrieval-augmented generation
Retrieval-augmented generation (RAG) is a pattern where a system first searches a collection of documents for passages relevant to a question, then gives those passages to a language model so its answer draws on them.
Document parsing
Document parsing converts files such as PDFs, scans, slides and spreadsheets into structured text that software can use, keeping headings, reading order, tables and images instead of a flat stream of characters.
Embedding
An embedding is a list of numbers (a vector) that represents a piece of text, an image or other data, produced so that items with similar meaning end up with vectors close together.
Vector database
A vector database stores embeddings alongside their source records and finds the stored vectors closest to a query vector, usually with an approximate index so search stays fast across millions of items.
Context window
A context window is the maximum amount of text, counted in tokens, that a language model can consider at once, covering the system prompt, conversation, documents, tool results and the response it writes.
Reranking
Reranking is a second search step that takes a short list of candidate documents and reorders them by relevance to the query, using a more accurate but slower model than the first search.

Sources