Language & reasoning models

What to look for

The questions that separate the options, in order of how much they matter.

1. 01

   Your task's benchmark, not the headline rank

   Decide first whether you need chat quality, coding, agent work or cheap bulk processing, because the leaders differ. A model that tops a blind chat vote can trail on long agent runs, and the reverse. Read each leaderboard below for the column that matches your workload, and treat any single 'best model' claim as marketing.

2. 02

   Artificial Analysis Intelligence Index: compare at the effort you will pay for

   Version 4.3 is a weighted average of 10 evaluations that Artificial Analysis runs itself under one harness: agents 30%, coding 20%, general knowledge 30% and scientific reasoning 20%, with four on private test sets. It is the best single independent number, but the same model appears several times at different effort settings (max, xhigh, high), and the top entry is usually the most expensive one. Read the Cost per Task, Median Tokens/s and First Chunk columns next to the score; a two-point gap is rarely worth double the bill.

3. 03

   Arena (formerly LMArena): a preference vote, with error bars

   Arena, at arena.ai since January 2026, ranks models from blind side-by-side votes by its users. Style control, which discounts length and markdown, has been the default since May 2025. Look at the 95% confidence interval and the rank range: models whose intervals overlap are statistically tied, whatever their position. Use the category tabs (coding, hard prompts, long queries, WebDev) and remember it measures what people prefer in a chat window, not whether an agent finishes a 40-step task.

4. 04

   Coding scores: SWE-bench Verified is no longer a clean signal

   OpenAI stopped reporting SWE-bench Verified in February 2026, citing contamination: models had seen the problems in training, and an audit of failures found many were caused by flawed tests. In July 2026 it also withdrew its recommendation of SWE-bench Pro after finding about 30% of the public tasks broken. Treat vendor SWE-bench numbers as weak evidence, check which agent scaffold was used, and give more weight to newer or independently run tests such as Terminal-Bench 4.0 inside the Artificial Analysis index, DeepSWE, and an evaluation on your own repository.

5. 05

   Agents and tool calls: read pass^k on tau-bench, then test your own tools

   Sierra's tau-bench, now in its third version (τ³-bench, at taubench.com), simulates customer-service agents using tools against a simulated user. The useful column is pass^k: the share of tasks solved on every one of k runs, which measures consistency rather than lucky single attempts. Rankings change as k grows, so a model that wins pass^1 may lose pass^4. Original tau-bench and τ³ scores are not comparable. Also check practical limits: some models need a specific API for tools (GPT-6.1 Sol requires OpenAI's Responses API), and strict JSON-schema output varies by provider.

6. 06

   Vendor launch tables versus independent runs

   Launch tables are run by the vendor, with settings it chose and rivals it picked. Independent runs often disagree: on Humanity's Last Exam in October 2026, the top score ranged from about 54% on Scale's closed-book board to about 65% on boards that allow tools. Before comparing two numbers, check who ran the test, whether tools were allowed, which effort level was used and whether the test set is public (and so possibly in training data).

7. 07

   Real price: output tokens, caching and expiring discounts

   List prices per 1M input and output tokens are the start, not the bill. Reasoning models spend output tokens thinking, so a cheaper model that thinks longer can cost more per task. Prompt caching changes the maths for agents and long system prompts: Claude Opus 5.5 charges $0.20 per 1M cached input against $4 uncached. Watch for long-prompt surcharges (OpenAI above 272K tokens), 50% batch discounts, and introductory prices that end (Gemini 4 Argon doubles on 1 December 2026).

8. 08

   Context window, speed and licence

   A 1M-token window says nothing about how well the model uses it; look at long-context tests such as AA-LCR and the output-token limit, which caps long generations. For interactive apps, time to first token and tokens per second on Artificial Analysis matter more than a point of benchmark score, and for open models they depend on the host. For open weights, read the licence: Apache 2.0 and MIT are unrestricted, while Kimi K3's and GLM-5.3's modified MIT licences add conditions for very large API hosts. Check hardware too: gpt-oss-120b fits one 80GB GPU, DeepSeek-V4-Pro needs a cluster.

12 tools, compared

Ranked by our rating. Open any review for sources, prices and the full verdict.

Side by side

The specifications that matter for this step. Checked 2026-10-23; confirm on the vendor’s site before you commit.

Head to head and ranked guides

• Best large language models (LLMs)
• Best AI reasoning models

Where this stage fits

• Chat with your documents: we would start with Claude Sonnet 5.5.
• Customer support agent: we would start with Claude Haiku 5.5.
• Coding agent: we would start with Claude Opus 5.5, GLM-5.3.
• Research & knowledge graph: we would start with Gemini 4 Argon.
• Assistant with long-term memory: we would start with GPT-6.1 Sol.
• Image generation pipeline: we would start with Gemini 3.8 Flash.
• Video production: we would start with Gemini 3.8 Flash.
• Voice agent: we would start with Gemini 3.8 Flash.
• Private, on-device AI: we would start with Gemma 4, Qwen3.8-27B.
• Document processing: we would start with GPT-6 Luna.
• Your own fine-tuned model: we would start with Qwen3.8-27B.
• Workflow automation: we would start with Claude Haiku 5.5.

Sources for this guide

• Artificial Analysis: Intelligence Index methodology
• Artificial Analysis: model leaderboard
• Arena leaderboard (formerly LMArena)
• Arena: LMArena is now Arena (28 January 2026)
• Arena leaderboard dataset notes on style control
• OpenAI: Why we no longer evaluate SWE-bench Verified
• OpenAI: Separating signal from noise in coding evaluations (SWE-bench Pro audit, July 2026)
• SWE-bench leaderboards
• Sierra tau-bench repository (τ³-bench)
• Artificial Analysis: Humanity's Last Exam results
• Z.ai: GLM-5.3 overview