Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

MTEB

MTEB, the Massive Text Embedding Benchmark, is a public benchmark and leaderboard that scores text embedding models across many task types, such as retrieval, clustering, classification and reranking, over many datasets and languages.

Also called: Massive Text Embedding Benchmark, MTEB leaderboard

By Divya Sharma · Checked Nov 15, 2026 · How we review

Before MTEB, embedding models were usually tested on a few datasets from one task, often semantic similarity. A model that did well there might do poorly at search. The MTEB paper introduced a broader test: in its first version it covered 8 embedding tasks, 58 datasets and 112 languages.

How it works

Each model turns the benchmark's texts into vectors, and each task scores those vectors in its own way. Retrieval tasks check whether the right documents rank near the top, using metrics such as nDCG and recall@k. Clustering tasks check whether similar texts group together. Reranking, classification, pair classification, semantic similarity and summarisation tasks complete the set. The leaderboard, hosted on Hugging Face, shows an average plus a score per task type.

How to read it

Look at the column for the task you care about, usually retrieval, not the overall average. Check the languages and domains: a model that tops English web text may be weak on legal contracts or Japanese. Compare model size, embedding dimension and maximum input length too, because they drive cost and speed. As with any public AI benchmark, some models may have trained on data close to the test sets, so confirm the shortlist on your own queries and documents. The embedding models stage guide walks through the columns.

Related terms

AI benchmark
An AI benchmark is a fixed, public set of tasks with an agreed scoring method, used to compare models on the same test, for example coding issues solved, exam questions answered or retrieval quality across many datasets.
Embedding
An embedding is a list of numbers (a vector) that represents a piece of text, an image or other data, produced so that items with similar meaning end up with vectors close together.
Retrieval metrics (recall@k, nDCG, MRR)
Retrieval metrics score how well a search system returns relevant documents. Recall@k checks how many relevant items appear in the top k, nDCG rewards ranking them high, and MRR measures how early the first correct one appears.
Reranking
Reranking is a second search step that takes a short list of candidate documents and reorders them by relevance to the query, using a more accurate but slower model than the first search.
Embedding dimension
Embedding dimension is the number of values in each vector an embedding model outputs. More dimensions can hold more detail, but every extra value adds storage, memory and search cost per item.

Sources