Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Test-time compute

Test-time compute is the amount of computation a model spends while answering a request, rather than during training. Letting a model think longer or try several answers can improve results on hard problems.

Also called: reasoning effort, thinking budget, inference-time compute, inference-time scaling

By Divya Sharma · Checked Nov 15, 2026 · How we review

For years, the main way to get a better model was to train a bigger one for longer. Test-time compute is the other lever: keep the model the same and let it do more work for each question. Ways to do that include writing out reasoning before answering, as reasoning models do, sampling several answers and picking the best, and checking answers with a verifier.

What the research shows

A 2024 paper by Snell and colleagues studied how much extra inference-time compute helps. It found that the best way to spend it depends on how hard the prompt is, and that on some problems a smaller model with more test-time compute could beat a much larger one. Extra thinking tends to help most on maths, coding and multi-step problems, and little on simple lookups.

Reasoning effort and thinking budgets

APIs expose this as a setting. Anthropic's manual extended thinking mode used a budget_tokens value for how many tokens Claude could spend reasoning; newer Claude models use an effort setting instead. Other providers offer similar low, medium and high reasoning effort options. These are the "reasoning effort" and "thinking budget" names you will see in docs.

The cost

Thinking tokens are usually billed as output tokens, so more test-time compute means a higher cost per request under token pricing and slower answers, including a longer time to first token for visible text. Set effort per task: low for routine requests, high only where tests show it helps. See our AI model pricing research for how prices compare.

How it differs from chain-of-thought

Chain-of-thought is one technique, prompting a model to show its steps. Test-time compute is the broader budget, which chain-of-thought is one way of spending.

Related terms

Reasoning model
A reasoning model is a language model trained to work through a problem in a stream of intermediate thinking tokens before giving its final answer, which tends to help on maths, coding and multi-step tasks.
Chain-of-thought
Chain-of-thought (CoT) is a prompting method in which a language model writes out intermediate reasoning steps before its final answer, which tends to improve results on maths, logic and other multi-step problems.
Token pricing
Token pricing is how most AI model APIs charge: a set price per million input tokens you send and a separate, usually higher, price per million output tokens the model generates, with discounts for cached input and batch jobs.
Output tokens
Output tokens are the tokens a language model generates in its reply. They are counted and billed separately from input tokens, and a max_tokens setting caps how many one response may contain.
Time to first token
Time to first token (TTFT) is how long a user waits between sending a request to a language model and receiving the first token of its reply, covering queuing, prompt processing and network time.

Sources