Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

LMArena

LMArena, first known as Chatbot Arena, is a public platform that ranks AI models by crowdsourced human votes: users see two anonymous models answer the same prompt and pick the better response.

Also called: Chatbot Arena, Arena, LMSYS Arena

By Divya Sharma · Checked Nov 15, 2026 · How we review

LMArena grew out of Chatbot Arena, an open research platform described in a 2024 paper, and its site now brands itself simply as Arena. The paper's authors call it one of the most referenced leaderboards for language models.

How it works

In battle mode, a user types a prompt and gets answers from two anonymous models. The user votes for the better one, and only then are the model names revealed. Hiding the names makes it a blind test, so brand does not sway the vote. The platform turns many such pairwise votes into ratings in the style of Elo, and publishes leaderboards for text, coding, vision, image, video and other tasks. The original paper reported over 240,000 votes at the time and found the crowd's votes agreed well with expert raters.

Why it is useful

Arena scores reflect what real users prefer on real prompts, not a fixed test that models can memorise. That makes it a useful complement to static benchmarks such as MMLU. The team also publishes open datasets of human preferences and research on methods such as style control.

Limits

Preference is not correctness. Voters can favour longer, confident or nicely formatted answers, and most prompts are short chats rather than long tasks. Labs can test pre-release models on the platform, and research has looked at how leaderboards can be gamed. Read a high Arena rank as "people liked its answers", and confirm with human evaluation or tests on your own use case.

Related terms

Elo rating (model arenas)
An Elo rating, in model arenas, is a score computed from many head-to-head votes between pairs of models, so that the gap between two scores predicts how often one model's answer will be preferred over the other's.
Leaderboard (AI)
An AI leaderboard is a public ranking of models by their scores on one or more benchmarks or on human preference votes, updated as new models are tested, so readers can compare models side by side.
Human evaluation
Human evaluation is judging AI outputs with people rather than automated metrics, for example raters scoring answers against a rubric or picking the better of two anonymous responses in a blind test.
LLM evaluation
LLM evaluation, often called evals, is measuring how well a model or AI application performs a task by running it on a set of test cases and scoring the outputs against defined criteria, by code, by humans or by another model.
AI benchmark
An AI benchmark is a fixed, public set of tasks with an agreed scoring method, used to compare models on the same test, for example coding issues solved, exam questions answered or retrieval quality across many datasets.

Sources