Skip to content
ReviewRamble

The best small and on-device AI models in 2026

The best small and on-device AI models right now are Gemma 4, Ministral 3 (3B, 8B and 14B) and MiniCPM5-2B, out of 5 we have reviewed. Each is ranked by our rating, with its price, our verdict and who it suits.

By Andrew Miller · Updated Oct 25, 2026 · How we review

How we ranked: by our rating out of 5, from reviews built on each product's own pricing, documentation and policies plus independent benchmarks where they exist, each checked on the date shown in the review. We take no affiliate commission.

  1. 1

    Gemma 4

    Gemma 4 is now the default open family below 32B parameters, because its licence finally matches its quality.

    Best for: Developers building on-device, local or single-GPU apps who want a permissive licence and multimodal input.

    Price: Free weights under Apache 2.0. Hosted on Google Cloud and third parties; OpenRouter lists 31B at $0.20/$0.40 and 26B A4B at about $0.08/$0.26 per 1M tokens.

    Read our Gemma 4 review
    4.4/5
  2. 2

    Ministral 3 (3B, 8B and 14B)

    A complete small-model family under a clean licence, with reasoning and vision in every size; Qwen still edges it on several of Mistral's own comparisons.

    Best for: Teams that need a small, permissively licensed model with vision for laptops, edge devices or cheap hosted inference.

    Price: Open weights under Apache 2.0. Mistral API: 14B $0.20, 8B $0.15, 3B $0.10 per 1M tokens, the same for input and output; batch half price.

    Read our Ministral 3 (3B, 8B and 14B) review
    4.1/5
  3. 3

    MiniCPM5-2B

    A 2B model that calls tools and patches code like a 4B one, and knows noticeably less than one.

    Best for: Developers putting tool-calling or coding helpers on phones, laptops and edge hardware, and researchers who want fully documented small-model training.

    Price: Free weights and training data under Apache 2.0.

    Read our MiniCPM5-2B review
    4.0/5
  4. 4

    gpt-oss-20b

    Fourteen months on, gpt-oss-20b is still the cheapest competent reasoning model to host, but Gemma 4 now does more in the same 16GB.

    Best for: Developers who want a free, private and fast reasoning model on a laptop or single consumer GPU, and do not need image input.

    Price: Free weights under Apache 2.0. Hosted from $0.018 input / $0.09 output per 1M tokens on OpenRouter; Artificial Analysis lists $0.07 / $0.25.

    Read our gpt-oss-20b review
    3.8/5
  5. 5

    SmolLM3-3B

    SmolLM3 is a glass-walled training run more than a product: unmatched for learning how a 3B model is made, outclassed for shipping one.

    Best for: Researchers and educators who want a small, fully reproducible model to study or fine-tune.

    Price: Free weights under Apache 2.0, with the training data recipe and checkpoints published. No hosted API on Hugging Face's own Inference Providers.

    Read our SmolLM3-3B review
    3.5/5

Sources

The main official source behind each ranked review; every review lists its full set.

More guides