What is Nemotron 3.5 Rerank 8B?
Nemotron 3.5 Rerank 8B is an open reranker from NVIDIA, released on 20 October 2026 under the OpenMDW-1.1 licence with 8.38 billion parameters. A reranker takes the shortlist a search system has already found and rescores each candidate by reading the query and the document together, which is slower than comparing vectors but more precise. This one accepts documents as text, page images or both, so charts, tables and scanned pages can be judged directly; queries are text.
It is a cross-encoder built on Mistral's Ministral 3 8B with a Pixtral image encoder, trained first on text and then on 2.3 million image samples, and is meant to sit behind NVIDIA's Nemotron 3.5 Embed. It outputs one raw score per pair, reads up to 32,768 tokens, and runs in Transformers, Sentence Transformers and vLLM on NVIDIA GPUs.
Is Nemotron 3.5 Rerank 8B worth it?
Nemotron 3.5 Rerank 8B is the strongest open reranker for visual documents. On the ViDoRe v3.1 reranking leaderboard, launched with MTEB, it ranked first on 20 October 2026 at 60.49 nDCG@10 on image-plus-text pages, against 59.17 for Alibaba's Qwen3-VL-Reranker-8B. In NVIDIA's own setup, reranking the top 50 results from Nemotron 3.5 Embed 1B, it lifts ViDoRe V3 image-plus-text from 58.82 to 67.56 and MIRACL-VISION from 68.93 to 78.69, against 66.03 and 75.06 for the Qwen reranker.
The gains carry into agent workloads. NVIDIA reports that adding the reranker cut an agent's token cost per query by about 14% with the 8B embedder and 13% with the 1B, because the model reading the results gets better pages first and searches less. Perplexity, in NVIDIA's account, found it scored the highest average on a private 72-task benchmark.
Plain text is less clear-cut. A community test posted under NVIDIA's announcement put it at 63.18 on BEIR against 65.11 for Qwen3-Reranker-8B, though ahead on MIRACL (75.06 against 74.13); NVIDIA publishes no text-only reranking table of its own. For text-only search, smaller rerankers such as Jina Reranker v3.5 at 0.6B or Cohere's hosted Rerank 5 remain the cheaper first choice.
The catch: only an 8B is offered, and it scores pairs one at a time, so reranking 50 candidates means 50 passes through an 8-billion-parameter model on every query, more for page images. The output is a raw logit rather than a calibrated probability, so thresholds have to be tuned per corpus. It runs only on NVIDIA GPUs under Linux, and it is free on OpenRouter for now with no published price for later.
Our take: put Nemotron 3.5 Rerank behind Nemotron 3.5 Embed when your corpus is PDFs and slides and precision justifies dedicated GPU time; for text search, a sub-1B reranker gets most of the benefit for a fraction of the compute.
Where it earns its place
An AI platform team at an insurer
Answer underwriters' questions from scanned policy documents and rate tables
Retrieve 50 pages with Nemotron 3.5 Embed 1B, rerank them with the 8B as image plus OCR text, and pass the top five to the language model; NVIDIA's image-plus-text ViDoRe V3 score rises from 58.82 to 67.56 with this setup.
A developer building a research agent that searches a document store repeatedly
Cut the tokens the agent spends reading irrelevant pages
Insert the reranker between search and the agent; NVIDIA measured about 14% fewer agent tokens per query with the 8B embedder. Prototype on OpenRouter's free endpoint before buying GPU time.
A search lead at a manufacturer with product manuals in many languages
Find the right diagram for a technician's question in Japanese or German
NVIDIA evaluated image retrieval in 18 languages including Japanese and German, so test on your own manuals; budget one GPU for the reranker, since every candidate page is scored separately.
Before you commit
- Setup
- pip install transformers 5.20.0 and sentence-transformers, load nvidia/Nemotron-3.5-Rerank-8B-BF16 and score query-document pairs; or serve with vLLM 0.32.0 in a separate environment.
- Integrations
- Transformers, Sentence Transformers, vLLM, NVIDIA NIM; OpenRouter (nvidia/nemotron-3.5-rerank-8b:free); pairs with Nemotron 3.5 Embed; Elastic support announced as coming soon.
- Data and privacy
- OpenMDW-1.1 allows commercial use and self-hosting, so documents stay on your hardware. Trained on 21 million text samples and 2.3 million image samples from commercially permissible and synthetic data.
- Cost at scale
- The weights and OpenRouter calls are free today; the real cost is GPU time, since an 8B model scores every candidate. NVIDIA lists A100 80GB, H100 80GB and GB200 as tested hardware; NIM pricing is not published.
- Limits
- 32,768 tokens per query-document pair including image tokens; images default to a 1,120-pixel longest edge, up to 1,540; text queries only; outputs raw logits; Linux and NVIDIA GPUs.
What’s good
- First on the ViDoRe v3.1 reranking leaderboard at 60.49 (20 October 2026)
- Judges page images, text or both
- Lifts MIRACL-VISION from 68.93 to 78.69 over Nemotron 3.5 Embed 1B in NVIDIA's test
- Permissive OpenMDW-1.1 licence; free on OpenRouter for now
What’s not
- 8B only; every candidate is a full forward pass
- Trails Qwen3-Reranker-8B on BEIR text in a community test
- Raw scores, not calibrated probabilities
- NVIDIA GPUs on Linux only
- No published text-only benchmark from NVIDIA
Best for
Teams running retrieval over PDFs, slides and scanned documents who need the most precise open second stage.
Nemotron 3.5 Rerank 8B alternatives
Nemotron 3.5 Rerank 8B: common questions
- Is Nemotron 3.5 Rerank 8B worth it?
- Nemotron 3.5 Rerank is the open reranker for document search where the answer may be in a chart, but at 8B it is a heavy second stage for plain text. We rate it 4.1 out of 5.
- How much does Nemotron 3.5 Rerank 8B cost?
- Free under OpenMDW-1.1. Free to call on OpenRouter for now; NVIDIA NIM available, with no public price.
- What are the downsides of Nemotron 3.5 Rerank 8B?
- 8B only; every candidate is a full forward pass; trails Qwen3-Reranker-8B on BEIR text in a community test; raw scores, not calibrated probabilities; nVIDIA GPUs on Linux only; no published text-only benchmark from NVIDIA.
- Who is Nemotron 3.5 Rerank 8B best for?
- Teams running retrieval over PDFs, slides and scanned documents who need the most precise open second stage.
- How much does Nemotron 3.5 Rerank 8B cost for a team or in production?
- The weights and OpenRouter calls are free today; the real cost is GPU time, since an 8B model scores every candidate. NVIDIA lists A100 80GB, H100 80GB and GB200 as tested hardware; NIM pricing is not published.
- Is Nemotron 3.5 Rerank 8B safe for company data?
- OpenMDW-1.1 allows commercial use and self-hosting, so documents stay on your hardware. Trained on 21 million text samples and 2.3 million image samples from commercially permissible and synthetic data.
- What are the best Nemotron 3.5 Rerank 8B alternatives?
- The closest alternatives we have reviewed are Cohere Rerank 5, Jina Reranker v3.5, Qwen3-Embedding and Qwen3-Reranker, zerank-2 and mxbai-rerank v2.
Answers checked Nov 7, 2026.
Make Nemotron 3.5 Rerank 8B? Show this rating on your site
