Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Diffusion language model

A diffusion language model (dLLM) generates text by starting from a fully masked or noisy sequence and refining many token positions in parallel over several steps, instead of writing one token at a time.

Also called: dLLM, masked diffusion language model

By Divya Sharma · Checked Nov 15, 2026 · How we review

Most large language models are autoregressive: they write left to right, one token per step. A diffusion language model borrows the approach used by image diffusion models. It begins with a sequence where every position is hidden, predicts all of them, keeps the predictions it is most confident in, and repeats until the text is complete.

How it works

The common recipe is masked diffusion. During training, random portions of real text are replaced with a mask token and the model, usually a transformer without the left-to-right mask, learns to fill them in. During generation it runs that process in reverse. The LLaDA paper describes exactly this: a forward masking process and a reverse generation process. LLaDA was trained with the same pretraining and supervised fine-tuning steps used for normal LLMs.

Why builders care

Because many tokens are filled at once, a diffusion model can produce text in fewer steps, which can mean much higher throughput. Google's experimental DiffusionGemma 26B A4B, reviewed on this site, is an example of that trade. The model can also look at the whole sequence while editing, which suits filling in the middle of code or text.

Limits

Quality per parameter has generally trailed the best autoregressive models, and fewer steps usually means lower quality. Serving tools, KV caching and streaming were built around token-by-token generation, so support for dLLMs is newer and thinner. Test on your own tasks before swapping one in.

Related terms

Diffusion model
A diffusion model is a generative model that learns to turn random noise into data, such as an image, by removing a little noise at a time over many steps.
Autoregressive model
An autoregressive model generates a sequence one piece at a time, predicting each next token from all the tokens before it and feeding that output back in. Most large language models work this way.
Large language model
A large language model (LLM) is a neural network with billions of parameters, trained on large amounts of text to predict the next token, which lets it write, answer questions, summarise and follow instructions.
KV cache
A KV cache stores the attention keys and values a transformer has already computed for earlier tokens, so each new token only needs work for itself. It speeds up generation but takes GPU memory that grows with context length.
Tokens per second
Tokens per second (tok/s) measures how fast a language model produces output. It is quoted either per user, the speed one reply streams at, or per system, the total output across all concurrent requests.

Sources