A careful, well-documented way to fit a 35B model on a 24 GB Mac or a phone, but it runs only two preview models of its makers' choosing and nearly every speed figure is their own. We rate it 3.6 out of 5.

How much does edge0 cost?

Free and open source under Apache 2.0, models included. No paid edition.

What are the downsides of edge0?

Runs only its own two preview checkpoints; no training code in the repo to adapt others; python framework is Apple Silicon only; CUDA and Linux are still promises; speed figures are almost all self-reported; apps are sideload-only: unsigned IPA, APK outside Google Play, ad-hoc signed Mac app.

How much does edge0 cost for a team or in production?

No paid edition, so the cost is hardware: the reference is a 24 GB Mac mini M4 Pro, and the paper's 16 GB M2 runs the 35B at 6.4 tok/s. The engine serves one request at a time, so more concurrent users means more machines.

Is edge0 safe for company data?

Runs locally after downloading from Hugging Face or ModelScope; the macOS app keeps models, chats and logs in ~/.edge0, and servers bind to 127.0.0.1, with a bearer token required for LAN access on macOS. No telemetry is described. Code and models are Apache 2.0; the bases are Qwen3.6-35B-A3B (Apache 2.0) and Ling-3.0-tiny-base (MIT).

What are the best edge0 alternatives?

The closest alternatives we have reviewed are Janas, llama.cpp, MLX and Ollama.

edge0 GitHub repo review (updated)

What is edge0?

edge0, from a company called Edge0, is an inference framework for mixture-of-experts language models that keeps the expert weights on an SSD and reads them in as each token needs them. Two additions make that fast enough to use: a prerouter, a small trained network per layer that guesses the next layer's experts one token ahead so the reads overlap the computation, and Recover-LoRA, an adapter distilled from the full-precision model that wins back most of the quality lost to 4-bit quantisation.

It runs the company's own two checkpoints, Edge0-35B-A3B-preview (built on Qwen3.6-35B-A3B) and Edge0-8B-A1B-preview (built on inclusionAI's Ling 3.0 tiny), published on Hugging Face and ModelScope with the adapters next to the weights. The Python package and its OpenAI-compatible server run on Apple Silicon Macs only; separate native engines power a macOS app, an iPhone app, an Android app and a Windows app, the last two on a patched llama.cpp. A unified framework across all of them is promised for Q4 2026, and the technical report is arXiv 2609.18063.

The repo was created on 8 September 2026, has about 8,800 GitHub stars and 1,370 forks, and is licensed Apache 2.0. Release v0.1.0 (21 October) ships a macOS app, an Android APK and an unsigned iPhone IPA; v0.1.1 (29 October) updates Android only.

Is edge0 worth it?

edge0 backs its SSD-streaming idea with a technical report, per-benchmark quality numbers and four native apps, and it is narrower than its star count suggests: it runs two checkpoints, both previews, both trained by the same team. The repo's one-line description promises 35B models in about 2.5 GB of RAM. The measured figure is 2.9 GiB, and it is the MLX allocator's peak, not what the process holds: on a 16 GB MacBook M2 the paper reports process memory of 4.91 to 6.39 GiB with the prerouter on, the rest being expert pages the operating system can reclaim. The README relabelled its column after an outside contributor pointed out the difference.

Almost every speed figure is the team's own. On the reference Mac mini M4 Pro with 24 GB, the README gives 14.9-17.7 tok/s for the 35B and 23.9-25.3 for the 8B; the paper, using a faster configuration, reports 20.4 tok/s against 3.9 for a plain mlx-lm server holding all 19.5 GB of weights in memory, which on that machine leaves too little room and pages. The prerouter matters most when the weights do not fit: on the 16 GB M2 the paper measures 3.5 tok/s rising to 6.4 at the released routing width, while on the 24 GB machine switching it off costs only 19.9 against 20.4. The README's "up to +59%" therefore depends on tight memory and slow storage. The platform READMEs add a MacBook Air M3 at 10-12 tok/s, an iPhone 16 Pro at 6.4 tok/s, and a 16 GB Snapdragon 8 Elite tablet at 6-9 tok/s, where the first turn on a new topic waits 10-15 seconds for experts to come off flash. Outside evidence is thin: one community post has the 8B at 14.55 and 27.17 tok/s on an M4 Pro, a report of 10.8-second waits before the first token on a 16 GB M1 Pro led to a fix, and we found no independent 35B measurement.

Quality is measured, which few local-inference projects offer, but by the team on its own OpenCompass runs. Across five benchmarks the int4 35B averages 79.2 against 83.2 for the fp16 Qwen3.6 base, and the 8B 69.9 against 72.7. The loss sits in long reasoning: AIME 2026 drops 6.1 points on the 35B and 10.0 on the 8B, while the 8B's MMLU-Pro comes out 4.3 points above its base. The model card calls tool use and multi-step agent work "currently weak" in this preview, and a Hugging Face request to verify the scores on Papers with Code is still open.

The SSD load is reads, and modest in bandwidth: at the released width the paper's 16 GB M2 moved 50.9 to 58.9 MiB per decode step, with the disk busy for at most 12% of each step, so the cost is waiting rather than wear. Prompt reading is the weak spot; the 8B's default once re-read 4.06 GiB of experts on every request until a --prefill-ondemand switch was added. The engine serves one request at a time, Android needs 12 GB or more for the 35B, and on an 8 GB phone the 8B fell to 1.14 tok/s. Distribution is developer-grade: the iPhone IPA is unsigned and must be re-signed with your own Apple ID (about 7 days on a free one), the APK is not on Google Play, the macOS app is ad-hoc signed and blocked by Gatekeeper until you clear the quarantine flag, and Windows has no binary release. On Windows the 35B's 27.7 tok/s comes from a 48 GB machine where the weights sit in RAM, and the team's own comparison puts its patches level with vanilla llama.cpp.

Against Janas, the closest rival, the two barely overlap. Janas runs Qwen, Gemma and Llama families converted from GGUF on x86-64 Linux and Windows, written by one author who cannot yet merge outside code; edge0 runs two models on Apple Silicon and phones, with 14 contributors and outside pull requests merged, tool calling among them. Neither has been measured on the other's hardware, and both publish mostly their own numbers. Who stands behind Edge0 is less clear than the code: the GitHub organisation, created in July 2026, gives its location as the United States, the five paper authors are named, and the website sells on-device models for earbuds, glasses, toys and robots, but its About page returned a 404 when we checked. The 1,370 forks against about 8,800 stars is a higher ratio than Janas's 44 against 548; the counts alone do not say why.

Where it earns its place

A developer with a 24 GB Mac mini M4 Pro
Run a 35B-class model locally and keep most of the memory free for other work
Install the python package, download the 23 GB Edge0-35B-A3B-preview checkpoint and run edge0 serve edge0-35b for an OpenAI-compatible endpoint on 127.0.0.1:8000. The README reports 14.9-17.7 tok/s at a 2.9 GiB allocator peak; first replies are slower while experts come off the SSD, and requests are served one at a time.
A mobile engineer at a consumer-app start-up
Prototype offline chat on an iPhone 16 Pro or a 16 GB Snapdragon 8 Elite Android device
Build the iOS app in Xcode under your own bundle ID with the model packed in (about 20 GB for the 35B), or sideload the 0.1.1 APK and import a converted GGUF. The team reports 6.4 tok/s for the 35B on the iPhone and 6-9 tok/s on the Android tablet; neither app is in a store, so this is a feasibility test, not a product.
A CTO at a 20-person firm weighing a private assistant on staff Macs
Decide whether edge0's models are safe to build on
Apache 2.0 on the code and weights, over bases licensed Apache 2.0 (Qwen) and MIT (Ling), allows commercial use. The risks are maturity: preview checkpoints the card calls weak at tool use, a unified framework still due in Q4 2026, and a one-request server, so budget one Mac per concurrent user and re-testing when the framework lands.

Before you commit

Setup: Python 3.10 or later (3.12 recommended) on an M1 or later Mac: pip install the python folder, fetch a checkpoint (about 23 GB for the 35B, 4.2 GB for the 8B) and run edge0 demo. The macOS app needs macOS 14 on M3 or later; Android needs arm64 and Android 13; Windows is build-from-source with Visual Studio 2022, the Vulkan SDK, Rust and Node, about 10-20 minutes for the engine.
Integrations: edge0 serve and the macOS daemon expose an OpenAI-compatible /v1/chat/completions with tool calling, plus a transformers-style Python API (AutoEngine). The Android and Windows engines are llama.cpp pinned at b11100 with edge0's patches; Windows serves through llama-server on loopback.
Data and privacy: Runs locally after downloading from Hugging Face or ModelScope; the macOS app keeps models, chats and logs in ~/.edge0, and servers bind to 127.0.0.1, with a bearer token required for LAN access on macOS. No telemetry is described. Code and models are Apache 2.0; the bases are Qwen3.6-35B-A3B (Apache 2.0) and Ling-3.0-tiny-base (MIT).
Cost at scale: No paid edition, so the cost is hardware: the reference is a 24 GB Mac mini M4 Pro, and the paper's 16 GB M2 runs the 35B at 6.4 tok/s. The engine serves one request at a time, so more concurrent users means more machines.
Limits: Only the two Edge0 checkpoints are registered; another model family needs an adapter plus its own trained prerouter heads and LoRA, and we found no training code in the repo. No CUDA or Linux build yet, agent and tool use are weak by the model card, and the unified framework due in Q4 2026 may change interfaces.

What’s good

Quality loss published per benchmark: 79.2 against 83.2 for the 35B
35B MoE at 14.9-17.7 tok/s on a 24 GB Mac mini, by the README
Native apps for macOS, iPhone, Android and Windows, plus an OpenAI-compatible server
Apache 2.0 for code and models; outside pull requests are merged

What’s not

Runs only its own two preview checkpoints; no training code in the repo to adapt others
Python framework is Apple Silicon only; CUDA and Linux are still promises
Speed figures are almost all self-reported
Apps are sideload-only: unsigned IPA, APK outside Google Play, ad-hoc signed Mac app

Best for

Apple Silicon owners with 16 to 24 GB who want a 35B-class model locally, and mobile developers testing on-device chat on recent phones, who accept preview models.

edge0 alternatives

Janas

Run models locallyC548 stars

3.6/5

Pure-C engine that runs mixture-of-experts LLMs larger than RAM by streaming experts from an NVMe SSD, on CPU, Vulkan GPU or Intel NPU.

llama.cpp

Run models locallyC++132k stars

4.8/5

C/C++ engine for running LLMs locally on CPUs, Apple silicon and most GPUs, home of the GGUF model format.

MLX

Run models locallyC++29k stars

4.4/5

Apple's array framework for machine learning on Apple silicon, now with a CUDA build; the engine behind mlx-lm and many Mac LLM apps.

Ollama

Run models locallyGo184k stars

4.6/5

One-command tool for downloading and running open LLMs locally, with a REST API and optional paid cloud models.

edge0: common questions

Is edge0 worth it?: A careful, well-documented way to fit a 35B model on a 24 GB Mac or a phone, but it runs only two preview models of its makers' choosing and nearly every speed figure is their own. We rate it 3.6 out of 5.
How much does edge0 cost?: Free and open source under Apache 2.0, models included. No paid edition.
What are the downsides of edge0?: Runs only its own two preview checkpoints; no training code in the repo to adapt others; python framework is Apple Silicon only; CUDA and Linux are still promises; speed figures are almost all self-reported; apps are sideload-only: unsigned IPA, APK outside Google Play, ad-hoc signed Mac app.
Who is edge0 best for?: Apple Silicon owners with 16 to 24 GB who want a 35B-class model locally, and mobile developers testing on-device chat on recent phones, who accept preview models.
How much does edge0 cost for a team or in production?: No paid edition, so the cost is hardware: the reference is a 24 GB Mac mini M4 Pro, and the paper's 16 GB M2 runs the 35B at 6.4 tok/s. The engine serves one request at a time, so more concurrent users means more machines.
Is edge0 safe for company data?: Runs locally after downloading from Hugging Face or ModelScope; the macOS app keeps models, chats and logs in ~/.edge0, and servers bind to 127.0.0.1, with a bearer token required for LAN access on macOS. No telemetry is described. Code and models are Apache 2.0; the bases are Qwen3.6-35B-A3B (Apache 2.0) and Ling-3.0-tiny-base (MIT).
What are the best edge0 alternatives?: The closest alternatives we have reviewed are Janas, llama.cpp, MLX and Ollama.

Answers checked Oct 31, 2026.