Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

VRAM

VRAM is the memory on a graphics card. For AI work it holds the model weights, the KV cache and working data, and whether a model fits in VRAM usually decides which GPU you need to run it.

Also called: GPU memory

By Divya Sharma · Checked Nov 16, 2026 · How we review

A GPU can only compute quickly on data in its own memory. If a model does not fit, it either fails to load or spills to slower system memory through CPU offloading, which cuts speed sharply. So "how much VRAM do I need?" is usually the first question when running a model yourself.

What uses VRAM when running a model

The weights come first: the number of parameters times the bytes per parameter. Lower precision or quantization shrinks this, for example from 16-bit to 4-bit weights. Next is the KV cache, which grows with context length and the number of simultaneous requests. A serving engine also needs some working space and overhead.

Training needs more

Training adds gradients, optimizer state and saved activations on top of the weights. Hugging Face's guide notes that Adam alone adds 8 bytes per parameter, and that batch size or sequence length can exhaust GPU memory even when the model itself fits. This is why methods such as LoRA and QLoRA exist.

Estimating it

Our GPU calculator estimates VRAM for a given model, precision and context length. Leave headroom rather than planning to fill the card. Macs and some newer chips use unified memory, where the CPU and GPU share one pool, which lets them load larger models than most consumer graphics cards at lower speed. If nothing local is big enough, rent a card from an AI GPU cloud.

Related terms

Quantization
Quantization is a technique that stores a model's weights, and sometimes activations, with fewer bits, such as 8 or 4 instead of 16, so the model needs less memory and often runs faster, at some cost in accuracy.
Model parameters
Model parameters are the numbers a neural network learns in training, mostly weights. The parameter count, written as 7B for seven billion, is the usual shorthand for a model's size.
KV cache
A KV cache stores the attention keys and values a transformer has already computed for earlier tokens, so each new token only needs work for itself. It speeds up generation but takes GPU memory that grows with context length.
GPU
A GPU (graphics processing unit) is a processor with many small cores built to run many calculations in parallel. Its speed at matrix maths makes it the standard hardware for training and running AI models.
Unified memory
Unified memory is a design in which the CPU and GPU share one pool of memory instead of each having its own. On Apple Silicon Macs and similar chips, it lets a local AI model use most of the system RAM as GPU memory.
Local LLM
A local LLM is a language model that runs on your own computer, phone or server instead of a provider's cloud API, so prompts and data stay on your hardware and it works without an internet connection.

Sources