Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

DwarfStar (ds4)

antirez's C inference engine for a handful of large open models, DeepSeek V4 Flash first, on 96 GB-plus Macs, DGX Spark and Strix Halo.

Reviewed by ReviewRamble editors · Updated

View on GitHub Open source

Share this page:XLinkedInFacebookRedditWhatsAppEmail

Our rating

4.1/5

ds4 gets frontier-class open models running at usable speed on a single big Mac or DGX Spark by refusing to be general, which is both why it is fast and why you may lose your model on the next commit.

What is DwarfStar (ds4)?

DwarfStar, published as ds4, is a small native inference engine written in C by Salvatore Sanfilippo (antirez, the creator of Redis). It deliberately supports only a few models: DeepSeek V4 Flash and V4.1 Flash, DeepSeek V4 PRO, GLM 5.2, 5.3 and 5.3 Flash, and Qwen3.8 Flash Next. It is not a general GGUF runner; it loads only the GGUF files the project itself produces.

The repository ships a multi-turn CLI (`ds4`), a native coding agent (`ds4-agent`), an HTTP server (`ds4-server`) with OpenAI-style chat and Responses endpoints plus an Anthropic-style messages endpoint, and `ds4-eval` for capability regression tests. Metal on Apple silicon is the primary target, with CUDA (DGX Spark and multi-GPU cards such as the L40S) and ROCm on Strix Halo. SSD streaming lets models larger than RAM run from disk, and two machines can share one model over RDMA or a layer pipeline.

Is DwarfStar (ds4) worth it?

ds4 is the most direct route to running DeepSeek V4 Flash on hardware a person can own. The README targets Macs with 96 GB or more for DeepSeek and GLM, 64 GB for Qwen3.8 Flash Next, and the starting download, Flash Q2, is about 81 GiB. The community site records 34.4 tokens per second of generation and 557 tokens per second of prefill at 32K context on a 128 GB M5 Max, and the README reports about 126 tokens per second aggregate across 16 sessions on eight L40S cards. The repository has about 24,400 GitHub stars since its first commit in May 2026.

The narrow scope is the design, not an accident. Instead of a generic loader, each model gets its own prompt templates, tool-call format, quantisation recipe and KV cache handling, and the project tests loading, tool calls, the server and the agent together. SSD streaming stretches the reach further: Qwen3.8's Q2 release keeps 95.37 GiB of n-gram tables on disk and only 41.73 GiB of weights in memory, so a fast local SSD becomes part of the hardware requirement.

The server listens on 127.0.0.1:8000 by default, and the docs say to put authentication and TLS in front before exposing it with --host 0.0.0.0; they describe no built-in authentication. Saved agent sessions live in ~/.ds4/kvcache and, as the README warns, may contain private information.

The catch: the README calls the software beta quality and says a model may be removed when a better one arrives, so a setup that works today is not a promise for next month. There are no tagged releases, only a moving main branch with about 873 open issues and pull requests. The code is written with heavy help from AI coding agents, which the project states openly, and its documented answer to missing features is to ask your own coding agent to add them.

Our take: for a developer with a 128 GB Mac who wants a local model good enough to replace a paid coding assistant for some work, ds4 is a better bet than a general runner today; anyone who needs a stable, supported server for a team should pin a commit and keep llama.cpp as the fallback.

Where it earns its place

  • A freelance developer with a 128 GB MacBook Pro

    Run Claude Code against a local model when the API bill or a client's data rules say no cloud

    Download Flash Q2 (about 81 GiB), start `ds4-server`, and point Claude Code's Anthropic base URL at it, following the client setup guide. The cost is the laptop and electricity; the community site records about 34 tokens per second of generation on an M5 Max.

  • A small ML team with a rack of L40S cards

    Serve DeepSeek V4 Flash to several internal users without buying newer GPUs

    Build with `make cuda-generic` and run the server with multiple sessions; the README reports about 126 tokens per second aggregate across 16 sessions on eight L40S cards. Put an authenticating proxy and TLS in front, since the docs describe neither.

  • A researcher comparing quantisation recipes

    Check that a model still answers correctly after aggressive 2-bit expert quantisation

    Run `ds4-eval` with the core or hard suite against each GGUF and compare traces; the repository also includes tools for building GGUF files and imatrix data.

Before you commit

Setup
Clone the repository, run `make` (or `make cuda-spark`, `make strix-halo`, `make cuda-generic`), then `./download_model.sh ds4f-q2`; the model download is the slow part at about 81 GiB.
Integrations
OpenAI chat and Responses plus Anthropic messages endpoints, with tools and SSE streaming; documented client setups for Pi, OpenCode, Codex CLI and Claude Code.
Data and privacy
Everything runs locally; the server binds to 127.0.0.1:8000 unless you pass --host. Saved sessions in ~/.ds4/kvcache can hold private conversation content. No telemetry is mentioned in the README.
Cost at scale
No licence cost. The real cost is hardware: 96 GB-plus of unified memory for DeepSeek or GLM, or several CUDA cards, plus fast SSD space for the larger GGUF files.
Limits
A fixed list of supported models that can change; no tagged releases; sessions with images cannot yet be saved; no documented authentication or TLS.

What’s good

  • Runs DeepSeek V4 Flash at about 34 tokens per second on a 128 GB Mac, per the project's figures
  • OpenAI- and Anthropic-compatible server works with Claude Code, Codex CLI and OpenCode
  • SSD streaming and two-machine RDMA stretch models past a single machine's RAM
  • Local-only by default: the server binds to 127.0.0.1

What’s not

  • Only the project's own GGUF files load; models can be dropped
  • Beta quality, no tagged releases, about 873 open issues and pull requests
  • Needs 64 GB at the very least and a fast SSD for the larger models
  • No documented authentication if you open the server to a network

Best for

Developers with a 96 GB-plus Mac, a DGX Spark or a Strix Halo box who want DeepSeek V4 Flash or GLM 5.3 Flash running locally behind their coding agent.

DwarfStar (ds4) alternatives

DwarfStar (ds4): common questions

Is DwarfStar (ds4) worth it?
ds4 gets frontier-class open models running at usable speed on a single big Mac or DGX Spark by refusing to be general, which is both why it is fast and why you may lose your model on the next commit. We rate it 4.1 out of 5.
How much does DwarfStar (ds4) cost?
Free and open source (MIT). No hosted or paid tier; you supply the hardware and download the weights.
What are the downsides of DwarfStar (ds4)?
Only the project's own GGUF files load; models can be dropped; beta quality, no tagged releases, about 873 open issues and pull requests; needs 64 GB at the very least and a fast SSD for the larger models; no documented authentication if you open the server to a network.
Who is DwarfStar (ds4) best for?
Developers with a 96 GB-plus Mac, a DGX Spark or a Strix Halo box who want DeepSeek V4 Flash or GLM 5.3 Flash running locally behind their coding agent.
How much does DwarfStar (ds4) cost for a team or in production?
No licence cost. The real cost is hardware: 96 GB-plus of unified memory for DeepSeek or GLM, or several CUDA cards, plus fast SSD space for the larger GGUF files.
Is DwarfStar (ds4) safe for company data?
Everything runs locally; the server binds to 127.0.0.1:8000 unless you pass --host. Saved sessions in ~/.ds4/kvcache can hold private conversation content. No telemetry is mentioned in the README.
What are the best DwarfStar (ds4) alternatives?
The closest alternatives we have reviewed are llama.cpp, Ollama and MLX.

Answers checked Nov 16, 2026.

Make DwarfStar (ds4)? Show this rating on your siteDwarfStar (ds4) rated 4.1 out of 5 on ReviewRamble