What is Needle 3?
Needle 3 is a tiny model from Cactus Compute, a Y Combinator-backed on-device AI company, released in September 2026. It does not chat. Given the functions an app exposes, it picks the right ones and fills in their arguments; given a schema, it extracts typed fields from messy text; and it returns sentence embeddings for local search. A grammar compiled from your schemas guarantees the JSON parses, and every answer carries a confidence score, below which the call is withheld.
The full model has 121 million parameters across 20 layers, 70.8 million of them in an n-gram memory table, compressed to about 2 bits. Every depth from 2 to 20 layers is a usable model cut from the same weights, so a product can ship a 29M, 4-layer version to a camera and the full model to a phone. Cactus says it was trained on 360 billion tokens of proprietary structured data.
Weights are under Apache 2.0, and LoRA fine-tuning runs locally through the Python package. The inference engine ships as prebuilt binaries, not source. Better Stack's 3 October video fine-tuned it to steer a virtual robot duck.
Is Needle 3 worth it?
Needle 3 is the smallest model we know of that does reliable tool calling for phone commands, and it is narrower than its marketing. On Mobile Actions, which maps phone requests to Android intents, the 121M model scores 86.0% exact-match against 82.4% for LFM2.5 1.2B and 88.4% for DeepSeek V4 Flash in the cloud. On DroidCall it leads every small model at 47.0%. Those are the results behind the claim that it beats models ten times its size.
Cactus also says it matches models two to three times bigger on extraction, and its own chart says otherwise. On DSTC8 it scores 40.7 F1 to 53.0 for the 230M LFM2.5, under twice its size, and on both SNIPS tests LFM2.5 350M leads it (34.0 to 30.2, and 29.0 to 24.7). On BFCL v4, the general function-calling benchmark, it scores 50.2% against 59.1% for LFM2.5 350M. All figures are Cactus's own, run through its shipped binary against rivals at f16.
The ladder is the clever part, and fine-tuning is where it pays. Cactus reports that tuning on DroidCall lifts every depth by 18 to 36 points, with the 4-layer, 29M model then passing DeepSeek V4 Flash on that task. Better Stack's duck demo makes the same case on a small scale: on 49 held-out prompts, a fine-tune lifted the full model from 27% to 84% and a 15 MB 4-layer version to 61%. Expect to fine-tune before shipping; the base model is a starting point.
The small print is worth reading. The engine is a prebuilt library with no published source, and the package and binaries send anonymous usage counts unless you set NEEDLE_TELEMETRY=0. The size figures also drift: the page says 8 to 29 MB, while the 20-layer needle3.cact on Hugging Face is 35.3 MB, and the release page claims 20.6k GitHub stars against about 14,400 on the repository itself.
Where it earns its place
A firmware team at a robot-vacuum maker
Turn spoken instructions like "clean the kitchen but leave the bedroom" into a sequence of commands
Fine-tune with LoRA on the product's own tool set, then needle build an 8-layer subnetwork for the robot's ARM board; pair with Whistle for speech. Route low-confidence calls to a confirmation prompt rather than acting.
An Android developer at a personal-finance app
Read bank notifications into merchant, amount and date fields on the phone
Declare a Pydantic schema and call extract(); output always parses. Benchmark against your own notifications first, since Cactus's chart shows 230M and 350M rivals ahead on some extraction tests.
A maker building a talking toy or desk robot
Add natural-language control with personality rules, offline
Better Stack fine-tuned it on an RTX 5090 to steer a simulated Open Duck Mini; on 49 unseen prompts accuracy rose from 27% to 84%. The 4-layer 15 MB build scored 61% and suits a smaller board.
Before you commit
- Setup
- pip install cactus-needle, decorate Python functions with @needle.tool and call Needle(tools=[...]).run(); the engine and weights download once. needle build --platform <folder> --layers N produces a device bundle.
- Integrations
- Python package; C API (needle_load, needle_init, needle_complete, needle_embed); a CLI with a local HTTP server; engines for 13 platform folders including iOS, Android, watchOS, MIPS cameras, the browser and WASI. Accepts OpenAI-style tool schemas. Loads Whistle in the same engine for speech input.
- Data and privacy
- Inference never touches the network and works air-gapped. Anonymous usage telemetry is on by default; NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1 disable it. Training data is proprietary and undisclosed.
- Cost at scale
- No licence or per-call fees. On a Raspberry Pi 5, Cactus reports decode from about 400 tokens a second at 20 layers to 4,000 at 2 layers. Platform fine-tuning and its 2-bit post-training are paid, at unpublished prices.
- Limits
- Tool schemas share a small context with the prompt, so large tool catalogues must be split or retrieved; the native API holds one model per process. Locally fine-tuned exports are 4-bit, larger than the shipped 2-bit file (63 MB at 20 layers in Better Stack's test).
What’s good
- 86.0% on Mobile Actions at 121M parameters, per Cactus
- Grammar-constrained output always parses; calibrated confidence per call
- One set of weights runs at any depth from 2 to 20 layers
- Local LoRA fine-tuning; Apache 2.0 weights
What’s not
- Trails 230M to 350M models on extraction and BFCL v4
- Engine is closed source; telemetry on by default
- Published sizes and star counts do not match the repositories
- Base model needs fine-tuning for most products
Best for
Device and app makers with a fixed, small set of actions who will fine-tune a model to run offline on very constrained hardware.
Needle 3 alternatives
Needle 3: common questions
- Is Needle 3 worth it?
- A credible phone-command model at a size nothing else reaches, sold with a broader claim than its own charts support. We rate it 3.6 out of 5.
- How much does Needle 3 cost?
- Free open weights under Apache 2.0; prebuilt engines free to download. The Cactus Platform sells 2-bit post-training, datasets and full fine-tunes; prices are not published.
- What are the downsides of Needle 3?
- Trails 230M to 350M models on extraction and BFCL v4; engine is closed source; telemetry on by default; published sizes and star counts do not match the repositories; base model needs fine-tuning for most products.
- Who is Needle 3 best for?
- Device and app makers with a fixed, small set of actions who will fine-tune a model to run offline on very constrained hardware.
- How much does Needle 3 cost for a team or in production?
- No licence or per-call fees. On a Raspberry Pi 5, Cactus reports decode from about 400 tokens a second at 20 layers to 4,000 at 2 layers. Platform fine-tuning and its 2-bit post-training are paid, at unpublished prices.
- Is Needle 3 safe for company data?
- Inference never touches the network and works air-gapped. Anonymous usage telemetry is on by default; NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1 disable it. Training data is proprietary and undisclosed.
- What are the best Needle 3 alternatives?
- The closest alternatives we have reviewed are LFM3, Gemma 4 and MiniCPM5-2B.
Answers checked Nov 2, 2026.