Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Small language model

A small language model (SLM) is a language model with relatively few parameters, usually a few billion or fewer, built to run on a laptop, phone or single GPU at lower cost than a large model.

Also called: SLM

By Divya Sharma · Checked Nov 15, 2026 · How we review

Small language models trade some breadth of knowledge for speed, low cost and the ability to run on local hardware. They suit tasks like classification, extraction, routing, autocomplete and on-device assistants, where a large language model would be slow or expensive.

How they get good

Small models lean on careful training data. Microsoft's Phi-3 report describes phi-3-mini, a 3.8 billion parameter model trained on 3.3 trillion tokens, that the authors say is small enough to run on a phone. Others are made by knowledge distillation, where a small student model learns from a larger teacher.

Running one

With quantization a small model fits in a few gigabytes of memory, so tools such as Ollama or llama.cpp can run it on a laptop. The GPU calculator shows how much memory a given size needs.

Limits

There is no agreed size boundary between small and large. Small models hold less world knowledge, handle long multi-step reasoning less well and are more sensitive to prompt wording. A common pattern is to send easy requests to a small model and hard ones to a bigger model.

Related terms

Large language model
A large language model (LLM) is a neural network with billions of parameters, trained on large amounts of text to predict the next token, which lets it write, answer questions, summarise and follow instructions.
Knowledge distillation
Knowledge distillation is a training method in which a smaller student model learns to imitate a larger teacher model's outputs, producing a distilled model that keeps much of the teacher's skill at lower cost.
Local LLM
A local LLM is a language model that runs on your own computer, phone or server instead of a provider's cloud API, so prompts and data stay on your hardware and it works without an internet connection.
Quantization
Quantization is a technique that stores a model's weights, and sometimes activations, with fewer bits, such as 8 or 4 instead of 16, so the model needs less memory and often runs faster, at some cost in accuracy.
Model parameters
Model parameters are the numbers a neural network learns in training, mostly weights. The parameter count, written as 7B for seven billion, is the usual shorthand for a model's size.

Sources