Chunking

What to look for

The questions that separate the options, in order of how much they matter.

1. 01

   Structure-aware splitting first

   Splitting on headings, sections and list boundaries keeps a definition or a procedure in one piece. This needs structure from the parser, so chunkers that read a parsed document (Docling's HybridChunker, Unstructured's by_title, Markdown header splitters) usually beat splitting raw text. Start here before trying anything clever.

2. 02

   Size measured in your embedding model's tokens

   Embedding models truncate input beyond their token limit, often silently, so chunk limits should be counted with the same tokenizer the model uses. Character-based limits (Unstructured's max_characters, LangChain's default splitter) only approximate this. Docling's HybridChunker and the token splitters in LangChain, LlamaIndex and Chonkie take a tokenizer directly.

3. 03

   Context carried with every chunk

   A chunk that says 'the limit is 30 days' is useless without knowing which policy it belongs to. Prefer chunkers that attach the document title, heading path and captions as metadata or prepend them to the text before embedding. A small overlap helps when a sentence straddles a boundary, but it is no substitute for headings.

4. 04

   Tables and code handled as units

   Generic text splitters cut tables mid-row and functions mid-body. Look for table-aware options, such as Docling repeating a table's header row in each piece or Unstructured isolating tables in their own chunks, and code splitters that follow syntax (LangChain, LlamaIndex, Haystack's Python AST splitter).

5. 05

   Be sceptical of semantic chunking

   Semantic chunkers split where embedding similarity between sentences drops. A 2024 study by Vectara researchers found the extra compute did not bring consistent gains over fixed-size chunks, and an August 2026 study of eight strategies found the best method changed with the embedding model, dataset and metric. Try it only after measuring a simple baseline. There is no public leaderboard for chunkers; anything you read is a single study or a vendor chart.

6. 06

   Measure retrieval, not chunk aesthetics

   Build a small test set of real questions with the passages that answer them, then compare recall at k (how often the right passage is in the top k results) for each chunking setting. Chunk size is a parameter to test, not a belief. Re-run the test whenever you change the parser or embedding model.

7. 07

   Maintenance and where the code lives

   Chunking code is small, so the risk is abandonment rather than bugs. LangChain archived its langchain-experimental package in May 2026, which is where its SemanticChunker lived, and a request to move it into the main splitters package was closed as not planned. Prefer chunkers inside a library you already depend on, or pin versions.

6 tools, compared

Ranked by our rating. Open any review for sources, prices and the full verdict.

Side by side

The specifications that matter for this step. Checked 2026-10-19; confirm on the vendor’s site before you commit.

Where this stage fits

• Chat with your documents: we would start with Chonkie.
• Customer support agent: we would start with Chonkie.
• Coding agent: we would start with LlamaIndex.
• Research & knowledge graph: we would start with Chonkie.
• Assistant with long-term memory: we would start with Chonkie.
• Voice agent: we would start with Chonkie.
• Private, on-device AI: we would start with Chonkie.
• Your own fine-tuned model: we would start with Chonkie.

Sources for this guide

• LangChain text splitters
• Sunsetting langchain-experimental (GitHub issue #87)
• LlamaIndex node parser modules
• Docling chunking concepts
• Unstructured chunking
• Haystack preprocessors
• feyninc/chonkie on GitHub
• Is Semantic Chunking Worth the Computational Cost? (arXiv)
• Chunking strategies study, August 2026 (arXiv)
• Late Chunking (Jina AI, arXiv)