ONNX
ONNX (Open Neural Network Exchange) is an open file format for machine learning models. It describes a model as a graph of standard operators so it can be trained in one framework and run in another.
Also called: Open Neural Network Exchange, .onnx
By Divya Sharma · Checked Nov 15, 2026 · How we review
Models are usually trained in a framework such as PyTorch, but the place you want to run them may be a phone, a browser, a server without Python, or a chip with its own runtime. ONNX is a common format between the two. You export the model once to a .onnx file and any runtime that supports ONNX can load it.
How it works
An ONNX file holds a computation graph: nodes for operations such as matrix multiply or softmax, the connections between them, and the model weights. The operators come from a versioned standard set called an opset, so a runtime knows exactly what each node means. ONNX Runtime is the most used engine, and it can hand parts of the graph to GPUs, NPUs and other accelerators.
Where it fits
ONNX is common for smaller models that need to run fast on many devices: embedding models, rerankers, vision and speech models, and models in browsers. It is less common for serving the largest LLMs, where dedicated engines such as vLLM or GGUF-based runners usually load weights directly.
How it differs from Safetensors
Safetensors stores only the weights; you still need code that defines the model. ONNX stores the weights and the graph of operations, so the file describes the whole model. That makes it portable, but it means export can fail on unusual layers, and models that change shape at run time can be awkward to export.
Limits
Not every runtime supports every operator or opset version, and quantized ONNX models may behave differently across hardware. Test the exported model against the original before you ship it.
Related terms
- Model weights
- Model weights are the learned numbers inside a neural network, set during training, that decide how it turns input into output. A checkpoint is a saved file or set of files containing those weights.
- Safetensors
- Safetensors is a file format from Hugging Face for storing model weights. Unlike Python pickle files, loading it cannot run hidden code, and it supports fast, memory-mapped loading of individual tensors.
- GGUF
- GGUF is a binary file format for storing a model's weights, tokenizer and metadata in one file, built for fast loading and local inference with GGML-based tools such as llama.cpp, often in quantized form.
- Inference
- Inference is running a trained model to produce output from new input, such as an LLM generating a reply to a prompt. It is the stage users pay for every time they call a model, as opposed to training.
- NPU
- An NPU (neural processing unit) is a chip, or a block inside a larger chip, built to run neural network maths quickly and with little power. Phones and laptops use NPUs to run small AI models on the device.
- Quantization
- Quantization is a technique that stores a model's weights, and sometimes activations, with fewer bits, such as 8 or 4 instead of 16, so the model needs less memory and often runs faster, at some cost in accuracy.