NPU
An NPU (neural processing unit) is a chip, or a block inside a larger chip, built to run neural network maths quickly and with little power. Phones and laptops use NPUs to run small AI models on the device.
Also called: neural processing unit, AI accelerator
By Divya Sharma · Checked Nov 15, 2026 · How we review
An NPU is a specialised processor for the matrix and vector operations that neural networks spend most of their time on. It can sit on its own, inside a CPU, or inside a GPU. Most people meet one in a phone or a recent laptop, where it handles tasks such as photo processing, speech recognition, background blur on video calls and small on-device language models.
Why it matters to builders
On consumer devices the NPU is designed to be small and power-efficient rather than fast at everything. That makes it a good fit for small language models and short, frequent jobs that should not drain the battery. It is a poor fit for large models: a laptop NPU does not have the memory or throughput of a data-centre GPU.
NPUs usually work in low-precision number formats such as INT8 or INT4, so models are normally quantized before they run on one. See numeric precision for what those formats mean.
How you program one
There is no single standard. Each chip maker ships its own toolkit, for example Apple Core ML, Intel OpenVINO, AMD Ryzen AI and Qualcomm's SDK, and a model often has to be converted into a format such as ONNX first. Vendors quote NPU speed in TOPS (trillions of operations per second), a figure that usually refers to INT8 operations and is hard to compare across chips.
NPU, GPU or TPU
A GPU is a general parallel processor that also happens to be good at AI. An NPU is narrower and aimed at efficient on-device inference. A TPU is Google's own accelerator design; the data-centre versions are rented through Google Cloud.
Related terms
- GPU
- A GPU (graphics processing unit) is a processor with many small cores built to run many calculations in parallel. Its speed at matrix maths makes it the standard hardware for training and running AI models.
- TPU
- A TPU (tensor processing unit) is Google's custom chip for machine learning, built around large matrix-multiply units and high-bandwidth memory. Google offers TPUs to other companies as a rented service on Google Cloud.
- Local LLM
- A local LLM is a language model that runs on your own computer, phone or server instead of a provider's cloud API, so prompts and data stay on your hardware and it works without an internet connection.
- Small language model
- A small language model (SLM) is a language model with relatively few parameters, usually a few billion or fewer, built to run on a laptop, phone or single GPU at lower cost than a large model.
- Quantization
- Quantization is a technique that stores a model's weights, and sometimes activations, with fewer bits, such as 8 or 4 instead of 16, so the model needs less memory and often runs faster, at some cost in accuracy.