Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Attention mechanism

An attention mechanism lets a neural network decide, for each token it processes, how much weight to give every other token in the input. Self-attention is the core operation inside a transformer.

Also called: self-attention, attention

By Divya Sharma · Checked Nov 15, 2026 · How we review

Attention is how a language model works out which earlier words matter for the word it is looking at now. In the sentence "the trophy did not fit in the case because it was too small", attention is the part that lets the model link "it" to "case".

How it works

Each token is turned into three vectors: a query, a key and a value. The model compares one token's query with every token's key to get a score, turns the scores into weights that add up to one, and then takes a weighted mix of the values. That mix becomes the token's new representation. Self-attention means the queries, keys and values all come from the same sequence. Models run several of these in parallel, called heads, so different heads can track different relationships.

Attention was used alongside recurrent networks before 2017. The paper that introduced the transformer showed that a network built from attention alone, with no recurrence, trained faster and translated better.

Why it matters when building

Attention compares every token with every other token, so its cost grows quickly as the input gets longer. This is one reason long context windows are expensive. During generation, models store past keys and values in a KV cache so they do not recompute them for each new token, and that cache is often what fills GPU memory. Techniques such as paged attention manage this memory, and alternatives such as state space models try to avoid the cost altogether.

Limits

Attention weights are sometimes shown as an explanation of what a model "looked at". They are a useful hint, not a full account of why the model produced an answer.

Related terms

Transformer (architecture)
The transformer is a neural network architecture, introduced in 2017, that processes sequences using attention instead of recurrence. It is the basis of nearly all current large language models and many image and audio models.
KV cache
A KV cache stores the attention keys and values a transformer has already computed for earlier tokens, so each new token only needs work for itself. It speeds up generation but takes GPU memory that grows with context length.
Context window
A context window is the maximum amount of text, counted in tokens, that a language model can consider at once, covering the system prompt, conversation, documents, tool results and the response it writes.
PagedAttention
PagedAttention is a method for storing a language model's KV cache in small fixed-size blocks that need not sit next to each other in GPU memory, like pages in an operating system. It cuts wasted memory so a server can run more requests at once.
State space model
A state space model (SSM) is a sequence model that carries a fixed-size hidden state forward as it reads each token, so its cost grows linearly with input length. Mamba is the best-known example.

Sources