Step 07 of 22
Retrieval & reranking
Combine keyword and vector search, then reorder results by relevance. The first search is fast but rough. A reranker reads the top results properly and puts the right passage first.
How to choose
- Add hybrid (keyword plus vector) search before adding a reranker.
- A reranker is worth its extra latency when the right answer is often in the top 20 but not the top 3.
- Keep the number of passages sent to the model small; more context is not better context.
What to look for
The questions that separate the options, in order of how much they matter.
01
Gain over your own first stage
A reranker's value depends on what it reranks. Voyage's own tests show rerank-3 lifting BM25 keyword search by 20% but a strong embedding model by only 6%. Measure recall at 100 and NDCG at 10 on your own queries with and without the reranker; if the right passage is rarely in the top 20, fix retrieval (hybrid search, better chunking) before adding one.
02
How to read BEIR and MTEB numbers
Most reranker scores are NDCG@10 on BEIR, a set of public retrieval datasets, or on the retrieval and reranking tasks of the MTEB leaderboard. A reranker's score depends on which first-stage retriever produced the candidates and how many it reranked (usually the top 100), and vendors choose different setups, so a 63 from one card and a 57 from another are not directly comparable. Prefer tables that run every model on the same candidates, look at the domain closest to yours (legal, code, multilingual MIRACL or Mr.TyDi) rather than the average, and treat vendor tables as claims to reproduce.
03
Latency per query
A reranker adds a model call to every search, and cost grows with candidates times document length. Published figures range from about 0.3 seconds (Jina v3.5 on an A100) to 0.9 seconds (mxbai-rerank-large-v2), and LLMs used as rerankers can take tens of seconds. Measure p95 with your real candidate count and passage length, and cut candidates from 100 to 30 or 50 if latency matters more than the last point of recall.
04
Cross-encoder, listwise or LLM reranker
A cross-encoder (pointwise) scores each query-document pair independently; most current models, including Qwen3-Reranker, zerank-2 and mxbai v2, are small LLMs used this way. Listwise rerankers such as Jina v3.5 and Mixedbread's v3.1 read the whole candidate list at once and can weigh documents against each other, but their context window must hold every candidate. Prompting a general LLM to rerank can match quality but is far slower and costlier per query.
05
Maximum input length
Older rerankers truncated at 512 tokens, silently scoring only the start of each passage. Current models accept 8K to 32K tokens per query-document pair, and Jina v3.5 takes 131K tokens for the query and all candidates together. Match the limit to your chunk size; if chunks are long, check whether the API splits documents and bills each piece, as Cohere does above 500 tokens.
06
Languages
Multilingual claims range from "multilingual" with no list to 100+ languages. Look for scores on MIRACL or Mr.TyDi and on cross-lingual queries (an English question over German documents), and test your own languages: mxbai v2, for example, trails BGE rerankers on Mr.TyDi despite its 100-language claim.
07
Licence and how you can run it
Qwen3-Reranker, zerank-2 and mxbai v2 are Apache 2.0 and can run anywhere. Jina v3.5 weights are non-commercial, so production self-hosting needs a paid Elastic licence. Voyage and Cohere are API-only, which removes GPUs from your stack but sends documents to a third party. Also check vendor stability: ZeroEntropy's hosted API was supported only until 4 September 2026 after Notion acquired the company in July, leaving only the open weights.
08
Price per 1,000 searches
APIs price per token (Voyage, Jina) or per search (Cohere). Convert to one unit: reranking 100 passages of 300 tokens is about 30,000 tokens a search, which at Voyage rerank-3's $0.05 per million tokens is about $1.50 per 1,000 searches, against about $2.50 per 1,000 for Cohere Rerank 4 Pro on gateway listings. Self-hosting trades that for GPU hours, which pays off only at steady, high volume.
How to test it on your own data
- Build 50 to 100 representative questions and mark the correct passages for each.
- Include hard cases: abbreviations, exact IDs, ambiguous questions, long passages and other languages you serve.
- Run your real first stage (hybrid if you use it) and record Recall@50 or @100; a reranker cannot recover passages that never reach it.
- Rerank those same candidates with each model and compare NDCG@10 and Recall@5 against no reranker.
- Measure p95 latency at your real candidate count, passage length and concurrency.
- Price each option per 1,000 searches, or in GPU hours if self-hosted.
- Check final answer quality end to end; a ranking gain that does not change answers is not worth its latency.
How the options differ
Retrieval happens in two stages. The first stage, keyword (BM25), vector or hybrid search, is fast and returns perhaps 50 to 100 candidates. A reranker then reads the query and each candidate together and reorders them, which is slower but far more precise, so only the best few passages reach the model. Frameworks such as LlamaIndex, Haystack and RAGFlow do not rerank themselves; they wire a retriever and a reranker together.
Rerankers split by how you run them and how they read. Hosted APIs (Voyage rerank-3, Cohere Rerank 4) need no GPUs and are priced per token or per search. Open-weight models (Qwen3-Reranker, zerank-2, mxbai-rerank v2, and Jina v3.5 under a non-commercial licence) run on your own hardware. Most score one document at a time; listwise models such as Jina v3.5 compare the whole shortlist in one pass.
9 tools, compared
Ranked by our rating. Open any review for sources, prices and the full verdict.
| Tool | Rating | Our verdict | Best for | Price |
|---|---|---|---|---|
| Voyage rerank-3model | 4.6/5 | On Voyage's own numbers rerank-3 leads every hosted rival, and the lite model costs so little that the rate limit, not the bill, decides how much reranking you do. | Teams running RAG or search in production who want the best hosted reranking quality for the money. | rerank-3 $0.05 per 1M tokens; rerank-3-lite $0.02 per 1M tokens; first 200M tokens free. |
| Jina Reranker v3.5model | 4.3/5 | A 0.6B reranker that beats a 4B one on BEIR, held back by a licence that turns self-hosting into a sales call. | Elastic users and teams that want a small, long-context reranker on their own hardware. | Jina API $0.05 per 1M tokens. Weights free for non-commercial use; commercial self-hosting needs a paid Elastic licence. |
| LlamaIndexrepo | 4.3/5 | Still a deep toolkit for document-heavy RAG, but its maker now says plainly that the framework is no longer its priority. | Python developers building RAG or document agents over large, varied data sources. | Framework free under MIT. LlamaParse (paid cloud): Free with 10k credits, Starter $50/month (40k credits), Pro $500/month (400k credits), Enterprise custom; 1,000 credits = $1.25. |
| Haystackrepo | 4.2/5 | A framework for people who want to read their RAG system rather than trust it, at the price of writing more of the wiring themselves. | Engineering teams building maintainable production RAG and agent pipelines in Python. | Free and open source under the Apache 2.0 licence. deepset Studio is free for one user; the Haystack Enterprise Platform is custom-priced. |
| Qwen3-Embedding and Qwen3-Rerankermodel | 4.2/5 | A complete open search stack under one licence, embedder and reranker together, though its leaderboard crown dates from June 2025. | Teams self-hosting multilingual or code search who want both embedding and reranking under one open licence. | Free under Apache 2.0. Qwen Cloud's hosted text-embedding-v4, built on Qwen3, is $0.07 per 1M tokens. |
| RAGFlowrepo | 4.0/5 | A finished document-to-answer product you can run on one server, whose security record says it belongs behind a VPN rather than on the open internet. | Teams that want a self-hosted document Q&A and knowledge assistant without building a pipeline. | Free and open source under the Apache 2.0 licence. You pay for the server and the model calls. |
| zerank-2model | 3.9/5 | A sharp open reranker whose calibrated scores are worth having, now with nobody behind it but your own GPU. | Teams already running GPU inference who want a free reranker with relevance scores they can use as a cut-off. | Free under Apache 2.0. ZeroEntropy's hosted API ($0.025 per 1M tokens at launch) was supported only until 4 September 2026 and now returns an unavailable error. |
| mxbai-rerank v2model | 3.8/5 | Still the safest small reranker to self-host on licence alone, though its benchmark lead dates from March 2025. | Teams that want a small, permissively licensed reranker to run on their own hardware. | Free under Apache 2.0. Hosted only inside Mixedbread Search: semantic search $4 per 1,000 queries, $7.50 with reranking; Scale plan $20/month in credits. |
| Cohere Rerank 4model | 3.7/5 | Rerank 4 still works, but Cohere's own Rerank 5 now beats it on every benchmark Cohere reports, which makes it a model to migrate from rather than adopt. | Teams with a tuned Rerank 4 pipeline who need time before re-validating on Rerank 5. | Billed per search (one query, up to 100 documents). Vercel AI Gateway lists Pro at $2.50 and Fast at $2.00 per 1,000 searches; Cohere's page no longer shows Rerank 4 rates. |
Side by side
The specifications that matter for this step. Checked 2026-10-23; confirm on the vendor’s site before you commit.
| Tool | Type | Parameters | Max tokens | Languages | Licence / access |
|---|---|---|---|---|---|
| Voyage rerank-3 | Hosted reranker, instruction-following | Not published | 32K query + document | Not published | Proprietary API, MongoDB Atlas |
| Jina Reranker v3.5 | Listwise LLM reranker | 0.6B | 131K for query + all candidates | 52 | CC BY-NC 4.0; API or paid licence |
| LlamaIndex | Framework (plugs in rerankers) | Not applicable | Depends on reranker | Depends on reranker | MIT |
| Haystack | Framework (plugs in rerankers) | Not applicable | Depends on reranker | Depends on reranker | Apache 2.0 |
| Qwen3-Embedding and Qwen3-Reranker | Pointwise LLM reranker | 0.6B, 4B, 8B | 32K | 100+ | Apache 2.0 |
| RAGFlow | RAG engine with hybrid + rerank | Not applicable | Depends on reranker | Depends on reranker | Apache 2.0 |
| zerank-2 | Pointwise reranker, calibrated scores | 4B | 32K | 100+ (vendor claim) | Apache 2.0; hosted API ended |
| mxbai-rerank v2 | Pointwise LLM reranker | 0.5B, 1.5B | 8K (32K compatible) | 100+ | Apache 2.0 |
| Cohere Rerank 4 | Hosted reranker (Pro and Fast) | Not published | 32K query + document | Multilingual (count not published) | Proprietary API, Model Vault |
Where this stage fits
- Chat with your documents: we would start with Voyage rerank-3.
- Customer support agent: we would start with Cohere Rerank 4.
- Research & knowledge graph: we would start with Jina Reranker v3.5.
- Private, on-device AI: we would start with Jina Reranker v3.5.