Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Autoregressive model

An autoregressive model generates a sequence one piece at a time, predicting each next token from all the tokens before it and feeding that output back in. Most large language models work this way.

Also called: next-token prediction, ARM, causal language model

By Divya Sharma · Checked Nov 15, 2026 · How we review

Autoregressive means the model's output becomes part of its next input. To write a sentence, the model predicts one token, appends it, and predicts again, until it produces a stop token or hits a limit. GPT-3, for example, is described by its authors as an autoregressive language model.

Next-token prediction

Next-token prediction is the training goal behind this. During pretraining the model sees huge amounts of text and learns to give a probability to every possible next token. At generation time, a sampling setting such as temperature picks one token from that distribution. A transformer uses a mask so each position can only attend to earlier positions, which keeps training consistent with how it generates.

What it means in practice

Because tokens come out one after another, response time depends on output length. That is one reason output tokens are often priced higher than input tokens, and why streaming is useful: you can show text as it arrives. Speed-ups such as speculative decoding and the KV cache all work around this one-at-a-time loop.

The model also cannot go back. Once a token is chosen it stays, so an early mistake can carry through the answer. Reasoning models partly work around this by writing out steps before the final answer.

The alternative

Diffusion language models start from a fully masked or noisy sequence and fill in many positions in parallel over several steps. The LLaDA paper challenges the common view that core LLM abilities depend on being autoregressive.

Related terms

Diffusion language model
A diffusion language model (dLLM) generates text by starting from a fully masked or noisy sequence and refining many token positions in parallel over several steps, instead of writing one token at a time.
Transformer (architecture)
The transformer is a neural network architecture, introduced in 2017, that processes sequences using attention instead of recurrence. It is the basis of nearly all current large language models and many image and audio models.
Token (LLM)
A token (LLM) is the basic unit of text a language model reads and writes, often a word, part of a word, a punctuation mark or a byte. Prices, speed and context limits are all counted in tokens.
Speculative decoding
Speculative decoding speeds up text generation by letting a small, fast draft model propose several tokens, which the large model then checks in a single pass, keeping the accepted ones. The output matches what the large model alone would produce.
Output tokens
Output tokens are the tokens a language model generates in its reply. They are counted and billed separately from input tokens, and a max_tokens setting caps how many one response may contain.

Sources