What is Strata?
Strata is a local inference engine built around a few large mixture-of-experts models: Qwen3.8-Flash-Next (125B parameters with 6B active, plus a 51B n-gram embedding), Qwen3.6-35B-A3B and DeepSeek V4.1-Flash. Instead of needing the whole model in graphics memory, it keeps the most-used experts on the GPU, holds every expert in system RAM where the CPU computes the ones the GPU lacks, and reads the model's 28.8 GB n-gram table from the SSD a few rows per token. The model's own multi-token prediction layer drafts tokens ahead.
It ships as a desktop app for Windows, Linux and macOS and as a terminal tool (`strata launch`), serves OpenAI- and Anthropic-compatible APIs on localhost, and includes a coding agent and image input. Supported hardware runs from NVIDIA RTX 20 to 50 series and recent AMD Radeon cards to Intel Arc on Linux, Apple silicon, Ryzen AI Max and, since 0.2.7.1, the NVIDIA DGX Spark.
The repository was created on 24 September 2026, is MIT-licensed, builds on parts of llama.cpp's ggml, and is almost entirely the work of one developer, Niko1221, with 3,594 commits against 29 for the next contributor.
Is Strata worth it?
Strata makes a frontier-sized open model practical on a gaming PC, but "any consumer hardware" oversells it. The ISTA-DASLab packs of Qwen3.8-Flash-Next need 37.6 GB (Q2_0) to 54.8 GB (IQ3_S) of RAM and VRAM combined, and the docs say RAM must cover the experts plus about 10 GB, so a 64 GB machine is the realistic floor. On that machine the numbers are striking: the project's own measurements on an RTX 5070 12 GB with a Ryzen 5 7600 show 93 tokens a second of output at 4K context with Q2_0 and 53 with the higher-quality IQ3_S.
Those figures are self-reported, but they are unusually well documented, and outsiders have added their own. A community report on a Radeon RX 9070 16 GB measured 58.7 tokens a second decoding IQ3_XXS at 4K, holding near 58 at 130K. Not every report flatters it: issue #2464, still open, describes Qwen3.6-35B-A3B decode on a V100 falling from 109 to 14.6 tokens a second at 114K context on the 0.2.0 beta, where the reporter says llama.cpp held about 90. Long prompts also take time: at 32K tokens the first word arrives after about 25 seconds.
The safety documentation is better than most commercial apps manage. It lists every network connection the code makes, states there is no telemetry, binds the server to 127.0.0.1, refuses to share it with other devices without an API key, and checks updates against an Ed25519-signed feed and model files against pinned SHA-256 hashes. The gaps are named too: the Windows installer is not code-signed, keys for outside model services sit unencrypted in a user-only file, and the bare engine server only warns if you open it to the network without a key.
The real risk is the shape of the project. One developer wrote nearly all of it in about five weeks, it has 507 open issues, and releases arrive every few days with long lists of per-GPU switches. llama.cpp supports far more models and hardware with a deep maintainer bench; Colibri, in pure C, also streams mixture-of-experts weights from disk. Strata's advantage is depth on a handful of models, which also means it is only as useful as Qwen3.8-Flash-Next and DeepSeek V4.1-Flash are to you.
Where it earns its place
A developer with an RTX 3060 or 5070 and 64 GB of RAM
Run a large coding model locally for Claude Code or OpenCode
Install the app or run `strata launch`, pick Q2_0 or IQ2_XS, then `strata connect` prints the settings for the agent. The project measures 79 to 93 tokens a second on an RTX 5070 at short context; the cost is the hardware and roughly 40 GB of model files.
A consultancy of ten handling client code that may not leave the office
Give staff a shared private assistant on one workstation
Run Strata on one PC with a 24 GB card and turn on Share with other devices, which will not start without an API key and exposes only the chat endpoints. Concurrent chats share one GPU's throughput; the project's DGX Spark tests found several chats mostly split the speed a single chat gets.
A hobbyist with a DGX Spark or a 128 GB Ryzen AI Max machine
Try DeepSeek V4.1-Flash without a server rack
Choose the 2.00 bpw set, a 257 GB download, and leave about 265 GB free on an NVMe drive. The project reports 19 to 21 tokens a second on code on Strix Halo; on 12 GB NVIDIA cards it is 4 to 11, because most of each token waits on the SSD.
Before you commit
- Setup
- Desktop installers for Windows, Linux and macOS from GitHub Releases, or a one-line shell or PowerShell script followed by `strata launch`. The model download (about 37 to 55 GB for Qwen3.8-Flash-Next) takes most of the time to a first answer.
- Integrations
- OpenAI- and Anthropic-compatible APIs on localhost; `strata connect` prints setup for Claude Code, Codex CLI, OpenCode, Hermes Agent, OpenClaw and others. CUDA, HIP, SYCL and Metal builds, plus Docker and source installs.
- Data and privacy
- No telemetry, according to the safety page, which lists every connection the code makes. The server binds 127.0.0.1; sharing needs an API key. Updates are Ed25519-signed and model files are checked against pinned SHA-256 hashes; outside-service keys are stored unencrypted. Strata is MIT, and Qwen3.8-Flash-Next is under the Qwen Community License 1.0.
- Cost at scale
- No paid edition; the project takes donations. The cost is a PC with a 12 GB or larger graphics card and about 64 GB of RAM for Qwen3.8-Flash-Next, plus around 265 GB of free NVMe space for DeepSeek V4.1-Flash.
- Limits
- 507 open issues and releases every few days. A few model families only; DeepSeek runs at 8K context; long-context decode on older GPUs has open regressions (issue #2464); speeds vary between runs because the cached experts change.
What's good
- Runs Qwen3.8-Flash-Next on a 12 GB GPU with 64 GB of RAM
- Detailed, reproducible speed reports from the author and the community
- No telemetry, localhost by default, signed updates and pinned model hashes
- OpenAI- and Anthropic-compatible APIs for Claude Code, Codex and OpenCode
What's not
- Needs roughly 40 to 55 GB of RAM plus VRAM, despite the any-hardware pitch
- Almost entirely one developer; 507 open issues
- Supports only a few model families
- Windows installer is not code-signed
Best for
Owners of a 12 to 24 GB graphics card and 64 GB or more of RAM who want a large open model for coding agents on their own machine.
Strata alternatives
Strata: common questions
- Is Strata worth it?
- A genuinely clever way to run a 125B-parameter model on a 12 GB graphics card, as long as you have 64 GB of RAM and can live with a one-person project that changes weekly. We rate it 4.0 out of 5.
- How much does Strata cost?
- Free and open source under the MIT licence; funded by donations. Model files carry their own licences (Qwen3.8-Flash-Next uses the Qwen Community License 1.0).
- What are the downsides of Strata?
- Needs roughly 40 to 55 GB of RAM plus VRAM, despite the any-hardware pitch; almost entirely one developer; 507 open issues; supports only a few model families; windows installer is not code-signed.
- Who is Strata best for?
- Owners of a 12 to 24 GB graphics card and 64 GB or more of RAM who want a large open model for coding agents on their own machine.
- How much does Strata cost for a team or in production?
- No paid edition; the project takes donations. The cost is a PC with a 12 GB or larger graphics card and about 64 GB of RAM for Qwen3.8-Flash-Next, plus around 265 GB of free NVMe space for DeepSeek V4.1-Flash.
- Is Strata safe for company data?
- No telemetry, according to the safety page, which lists every connection the code makes. The server binds 127.0.0.1; sharing needs an API key. Updates are Ed25519-signed and model files are checked against pinned SHA-256 hashes; outside-service keys are stored unencrypted. Strata is MIT, and Qwen3.8-Flash-Next is under the Qwen Community License 1.0.
- What are the best Strata alternatives?
- The closest alternatives we have reviewed are llama.cpp, colibri and Ollama.
Answers checked Oct 28, 2026.
Make Strata? Show this rating on your site
