The open-model shortlist, audited

Shortlists of open-weight models age fast. We took ten that keep turning up on them and checked each against its model card and the vendor’s own pages. Several have been retired or overtaken.

Still worth testing

• GLM-5.3-Flash (4.3): Z.ai’s MIT-licensed mixture-of-experts model with image and video input and a 1M-token context, at $0.15 per million input tokens and $0.50 output on Z.ai’s API. Its benchmarks are Z.ai’s own, and the weights still need a multi-GPU server.
• Gemma 4 (4.4): the E2B and E4B sizes are Google’s picks for phones and edge devices, and they take audio as well as images. Apache 2.0.
• Granite 4.2 (4.0): IBM’s 3B, 8B and 30B dense reasoning models under Apache 2.0. Text only, and every benchmark is IBM’s own.

Fine for narrow jobs

• Falcon-H1R-7B (3.6): very strong at maths for a 7B model, weak at agent tasks. The Falcon LLM licence asks for attribution and an acceptable-use policy, so it is not as simple as Apache 2.0. The related Falcon-H1-3B-Base is a 2025 base model we would not add to a test today.
• SmolLM3-3B (3.5): the most transparent small model, with its full training recipe published. Newer 3B and 4B models beat it on ability.
• Devstral Small 2 (3.6): a 24B Apache 2.0 coding model that runs on one RTX 4090. Mistral retired the API version in March 2026, so you will be self-hosting.

Overtaken or retired

• Qwen3.5-397B-A17B (3.8) has been superseded by Qwen3.6 and Qwen3.8.
• Devstral 2 (3.1) was retired from Mistral’s API in July 2026, which points users to Mistral Medium 3.5. Its modified MIT licence also excludes companies with over $20 million in monthly revenue.
• Phi-4-multimodal (3.3) is still the only Phi model that hears speech, but it dates from February 2025 and Gemma 4’s small sizes are the stronger pick now.

How to test any of them

Licences differ inside a single family: Devstral Small 2 is Apache 2.0, Devstral 2 is not. Read the model card for the exact checkpoint, check it fits your hardware, then run it on your own tasks. A small parameter count tells you what it costs to run, not whether its answers are good enough.