Embeddings

What to look for

The questions that separate the options, in order of how much they matter.

1. 01

   Retrieval quality on your own questions

   The only question that matters is whether a query brings back passages that actually answer it. On the MTEB leaderboard that is the Retr. column, not Mean (Task) or the Borda rank, which mix in classification, clustering and sentence-similarity tasks a RAG system never runs; Octen-Embedding-8B ranks seventh overall on the multilingual tab yet scores higher on retrieval than the fourth-placed Qwen3-Embedding-8B (71.61 against 70.88). Use benchmarks to shortlist; use your own retrieval evaluation to choose.

2. 02

   Fit with your domain

   General benchmarks say little about contracts, clinical notes or part numbers. Check how candidates handle your technical terms, abbreviations and closely related concepts that must stay apart, such as two similar product codes. The leaderboard's domain tabs (Code, Legal, Medical, Finance, Chemical) are a better guide than the overall score, and models that accept task instructions often gain a few points when given one.

3. 03

   Language support, including cross-language search

   Test both same-language retrieval (a German question over German documents) and cross-language retrieval (an English question over Japanese documents) if your users need it. On MTEB, compare the Multilingual tab (MMTEB, 131 tasks across more than 250 languages) with the English tab and any per-language tab for your markets; a model's English score can flatter it. A claim of '100+ languages' says nothing about quality in any one of them.

4. 04

   Input length that fits your chunks

   The maximum input (Max tokens on the leaderboard) must cover your longest chunk plus any title or heading you prepend, or text is cut off, often silently: all-MiniLM-L6-v2 stops at 256 word pieces. A larger limit does not automatically help, since one vector for a long passage blurs several topics together; it mainly gives you room to choose chunk size freely.

5. 05

   Dimensions: storage cost, not a quality score

   More dimensions mean more storage, memory and search compute per vector, and do not guarantee better retrieval. Many current models use Matryoshka training, so you can keep only the first 256 or 512 numbers with modest loss: Gemini Embedding 2 runs from 128 to 3,072 and renormalises truncated vectors automatically, while EmbeddingGemma 2 asks you to renormalise yourself. Int8 or binary output, offered by Voyage and Cohere, cuts storage further; test recall at the size you will actually store.

6. 06

   Speed and cost across the whole lifecycle

   Count four costs: embedding the initial corpus, re-embedding updates, query latency, and hosting or API fees. API prices for text run from $0.02 to $0.20 per million tokens (Jina v5 nano to Gemini Embedding 2), with batch discounts of 33 to 50% at Voyage, OpenAI and Google. Open models avoid fees but need GPUs sized to the leaderboard's Memory (MB) figure: about 1.1 GB for Qwen3-Embedding-0.6B against 14.4 GB for the 8B.

7. 07

   Deployment and privacy

   Decide early whether documents and queries may leave your infrastructure. If not, you need open weights you can run yourself, and the licence must allow your use: Qwen3-Embedding and EmbeddingGemma 2 are Apache 2.0, while Jina v5 and Nemotron ColEmbed V2 weights are non-commercial. Hosted APIs remove operations work but add a vendor that sees every query.

8. 08

   Supported inputs

   Text-only models need everything converted to text first. If your corpus includes scanned pages, slides, photos, audio or video, consider a multimodal model that embeds them directly (see the guide by data type below), and check it was benchmarked on that input type: MTEB has separate tabs for images (MIEB), visual documents (ViDoRe), audio (MAEB) and video (MVEB).

9. 09

   Usage requirements that break retrieval if ignored

   Read the model card before indexing. Many models expect a query prefix or instruction and a different (or no) prefix for documents; some need vectors normalised and specify cosine or dot-product similarity; EmbeddingGemma 2 warns that float16 silently corrupts output. Query and document vectors must come from the same model and embedding space, even when dimensions match, so changing models generally means re-embedding everything; Voyage 4's shared space across sizes is a rare exception.

Which approach for which data

Different content needs different models. Many systems use more than one and route by data type.

Reading the MTEB leaderboard

For RAG and document search, start with the Retr. column on the tab that matches your languages, not the overall mean; these columns rate embedding models, not vector databases.

Rank (Borda)

Each task 'votes' for models by their relative position on it; the model with the most votes across all tasks ranks first.

Rewards broad consistency, not retrieval. Two adjacent ranks can hide a large or a negligible gap, so look at the scores.

Parameters (total and active)

Model size, including embedding weights. Active parameters differ only for mixture-of-experts models.

A proxy for memory and speed. The table's Pareto tag marks models no smaller rival beats; check it before paying for an 8B model.

Openness

A 0–6 score for open weights, open licence, open training code, open training data, a paper and a model card.

Open weights alone do not mean commercial use is allowed; read the licence. Closed APIs score low here by design.

Zero-shot %

The share of the benchmark's tasks whose training data the model was not trained on. 'NA' means unknown.

Not an accuracy score. Below about 80%, part of the score reflects tasks the model has seen; on the English tab several top-five models sit at 48% to 51%.

Mean (Task)

Plain average across every task in the benchmark.

Easy to compare but weighted towards whichever task types have the most datasets: on the multilingual tab, 43 of 131 tasks are classification and 18 are retrieval.

Mean (TaskType)

Average within each category (classification, retrieval and so on), then an equal-weight average of the categories.

Fairer across categories than Mean (Task), but still includes tasks irrelevant to search. Expect it to run several points lower.

Mean (Public) and Mean (Private)

Averages over the openly released tasks and over held-out tasks the benchmark keeps private, on benchmarks such as RTEB that include both.

A model far better on public than private tasks may be tuned to the benchmark. Private scores are the more honest signal.

Retr.

Retrieval: find the relevant documents for a query in a large corpus, mostly scored by nDCG@10 (how high the right documents appear in the top ten).

Prioritise this column for RAG. Then confirm on the domain and language tabs closest to your data.

Rerank

Re-order a short candidate list for a query.

Useful secondary signal for search. A dedicated reranker model usually beats an embedder here.

Class. and MultiClass.

Classification: assign one or several labels from a fixed set, using the embeddings as features.

Relevant if you will tag or route content with embeddings; not a retrieval measure.

Clust.

Clustering: group texts without labels; scored against gold groups.

Relevant for topic discovery and deduplication, not for answering questions.

PairClass.

Pair classification: decide a relation between two texts, such as duplicate or paraphrase.

A guide for duplicate detection. Says little about query-to-passage retrieval.

STS

Semantic textual similarity: rate how similar two sentences are on a scale.

A high STS alone does not establish good retrieval; questions and answers are not paraphrases of each other.

Bitext

Bitext mining: match sentences with their translations across languages.

A rough signal for cross-language alignment; still test cross-language retrieval directly.

Summ.

Summarisation (English tab): score how well embeddings rank machine summaries against human references.

Evaluates summaries; it does not measure a model's ability to generate them, and matters little for search.

Instr Rerank

Reranking guided by a free-form instruction attached to the query.

Most models score between 0 and 12 here; ignore it unless you plan instruction-driven search.

Embedding dimension

Length of the vector the model outputs (the leaderboard lists the default or maximum).

A storage cost, not a quality score. Check whether smaller Matryoshka sizes are supported.

Max tokens

The longest input the model reads in one go.

Must exceed your chunk size; beyond that, bigger is not better for retrieval.

Memory (MB)

Memory needed to load the weights, in the CSV download and on each model's page.

Size your GPU or CPU from this. Blank for closed APIs.

11 tools, compared

Ranked by our rating. Open any review for sources, prices and the full verdict.

Side by side

The specifications that matter for this step. Checked 2026-10-19; confirm on the vendor’s site before you commit.

Where this stage fits

• Chat with your documents: we would start with Qwen3-Embedding and Qwen3-Reranker.
• Customer support agent: we would start with Cohere Embed 5.
• Coding agent: we would start with Qwen3-Embedding and Qwen3-Reranker.
• Research & knowledge graph: we would start with voyage-4-large.
• Assistant with long-term memory: we would start with Jina Embeddings v5.
• Image generation pipeline: we would start with Qwen3-VL-Embedding.
• Video production: we would start with Marengo 3.5.
• Voice agent: we would start with EmbeddingGemma 2.
• Private, on-device AI: we would start with EmbeddingGemma 2.
• Document processing: we would start with Qwen3-VL-Embedding.

Sources for this guide

• MTEB leaderboard
• MTEB leaderboard API
• MTEB leaderboard release post (June 2026)
• MTEB docs: running the leaderboard and zero-shot annotation
• Leaderboard column definitions (leaderboard-frontend)
• MMTEB: Massive Multilingual Text Embedding Benchmark (arXiv)
• Gemini API: embeddings (Matryoshka and normalisation)
• Voyage AI docs: embeddings
• illuin-tech/colpali on GitHub
• Qwen3-VL-Embedding-8B model card
• Marengo 3.5 (TwelveLabs)