TPU
A TPU (tensor processing unit) is Google's custom chip for machine learning, built around large matrix-multiply units and high-bandwidth memory. Google offers TPUs to other companies as a rented service on Google Cloud.
Also called: tensor processing unit, Cloud TPU
By Divya Sharma · Checked Nov 15, 2026 · How we review
Most AI work runs on GPUs, which began as graphics chips. Google designed the TPU as an application-specific chip that does one job: the matrix maths at the heart of neural networks. Its core is a matrix unit built as a systolic array, a grid of multiply-and-add cells that pass data to their neighbours, paired with on-chip high-bandwidth memory.
How you use one
You cannot buy a data-centre TPU. You rent it on Google Cloud, either as single machines or as "slices" of many chips linked by a fast interconnect for large training runs. Code is compiled for the TPU by the XLA compiler, which JAX, PyTorch (through PyTorch/XLA) and TensorFlow can target.
Why it matters
TPUs are one of the few large-scale alternatives to NVIDIA hardware. For a builder, they are an option when GPU supply is tight or TPU pricing suits a workload, especially for teams already working in JAX.
TPU versus GPU and NPU
A GPU is a general parallel processor with a huge software ecosystem built around CUDA. A TPU is narrower and tied to Google Cloud and XLA, so some libraries and custom kernels written for GPUs will not run on it without work. An NPU is the small, low-power equivalent found in phones and laptops for on-device inference.
Limits
Debugging and performance tuning follow different rules from GPUs, and model support is narrower in some open-source tools. Check that your framework and model run on TPUs before committing.
Related terms
- GPU
- A GPU (graphics processing unit) is a processor with many small cores built to run many calculations in parallel. Its speed at matrix maths makes it the standard hardware for training and running AI models.
- NPU
- An NPU (neural processing unit) is a chip, or a block inside a larger chip, built to run neural network maths quickly and with little power. Phones and laptops use NPUs to run small AI models on the device.
- GPU cloud
- A GPU cloud is a service that rents out GPU machines by the hour or second for AI training, fine-tuning and inference. Specialist providers, sometimes called neoclouds, focus on GPUs and often charge less than the large general clouds.
- Model serving
- Model serving is running a trained model as a service that applications call over a network, with software that loads the weights onto GPUs, queues and batches requests, manages memory and returns results.
- Inference
- Inference is running a trained model to produce output from new input, such as an LLM generating a reply to a prompt. It is the stage users pay for every time they call a model, as opposed to training.