GPU
A GPU (graphics processing unit) is a processor with many small cores built to run many calculations in parallel. Its speed at matrix maths makes it the standard hardware for training and running AI models.
Also called: graphics processing unit
By Divya Sharma · Checked Nov 16, 2026 · How we review
GPUs began as chips for drawing 3D graphics, where the same calculation runs on millions of pixels at once. NVIDIA's CUDA, introduced in 2006, opened that parallel throughput to general computing. Neural networks turned out to fit this well, because training and running them is mostly large matrix multiplications that split naturally across many cores.
Why AI needs GPUs
A CPU has a few powerful cores designed for varied, sequential work. A GPU trades that for very many simpler cores and very fast memory, which suits the repeated arithmetic in a transformer. Modern data-centre GPUs add dedicated units for low-precision maths, which is why formats such as FP8 and 4-bit matter.
What to look at
For running models, memory often matters more than raw compute. VRAM size decides which models fit, and memory bandwidth largely sets tokens per second during generation. For training and large models, the links between GPUs matter, since work is split across many cards with methods like tensor parallelism. Our GPU calculator estimates what a model needs.
Alternatives
Google's TPU is a chip designed specifically for machine learning, used mainly in Google Cloud. NPUs in phones and laptops run small models at low power. Apple and some other chips use unified memory shared by CPU and GPU. You can also skip owning hardware and rent from an AI GPU cloud or a serverless GPU platform.
Related terms
- VRAM
- VRAM is the memory on a graphics card. For AI work it holds the model weights, the KV cache and working data, and whether a model fits in VRAM usually decides which GPU you need to run it.
- TPU
- A TPU (tensor processing unit) is Google's custom chip for machine learning, built around large matrix-multiply units and high-bandwidth memory. Google offers TPUs to other companies as a rented service on Google Cloud.
- NPU
- An NPU (neural processing unit) is a chip, or a block inside a larger chip, built to run neural network maths quickly and with little power. Phones and laptops use NPUs to run small AI models on the device.
- GPU cloud
- A GPU cloud is a service that rents out GPU machines by the hour or second for AI training, fine-tuning and inference. Specialist providers, sometimes called neoclouds, focus on GPUs and often charge less than the large general clouds.
- Unified memory
- Unified memory is a design in which the CPU and GPU share one pool of memory instead of each having its own. On Apple Silicon Macs and similar chips, it lets a local AI model use most of the system RAM as GPU memory.
- Tensor parallelism
- Tensor parallelism splits each layer of a neural network across several GPUs, so that every GPU holds part of each weight matrix and computes its share of every step. It lets a model too large for one GPU run across many.