What is MobileMoE (S, M, L)?
MobileMoE is a family of on-device mixture-of-experts language models from Meta's FAIR research group, described in a paper on 26 May 2026 and posted to Hugging Face on 10 and 11 June 2026, with the weights refreshed in late August. A mixture-of-experts model holds many small feed-forward blocks and runs only a few of them for each token, so it can store more knowledge than it spends compute on. MobileMoE applies that idea at phone scale.
There are three sizes. S has 272 million active parameters out of 1.3 billion, M 528 million out of 2.8 billion and L 922 million out of 5.3 billion; each routes every token to 4 of 60 small experts plus one shared expert that is always on. Every size comes as a Base model, an instruction-tuned SFT model and a QAT model trained for 4-bit weights, nine checkpoints in all, at 0.68, 1.48 and 2.75 GB of INT4 weights. They read and write English text only, with an 8,192-token context.
Meta trained them on about 6 trillion tokens of open-source data, then mid-training, fine-tuning and quantisation-aware training, and ran them on a Samsung Galaxy S25 and an iPhone 16 Pro through a custom MoE kernel in ExecuTorch. The weights are under the FAIR Noncommercial Research License, so this is a research release, not a model a company can ship in a product.
Is MobileMoE (S, M, L) worth it?
MobileMoE is the clearest evidence yet that mixture-of-experts pays at phone scale, and it arrives under a licence that confines it to the lab. On Meta's re-run of 14 foundational benchmarks the instruction-tuned L model averages 60.3, against 50.8 for Qwen3.5 2B, 50.5 for SmolLM2 1.7B and 55.6 for AI2's OLMoE-1B-7B, which spends 1.3 billion active parameters to MobileMoE-L's 922 million. Meta ran every model itself under identical settings, so these are self-reported figures, not an independent leaderboard.
The gains are uneven. Maths and code are where L pulls ahead: 76.8 on GSM8K against Qwen3.5 2B's 61.3, 39.4 on MATH-500 against 31.0, and 65.2 on HumanEval against 50.0. Instruction following is the weak spot. Qwen3.5 2B scores 73.3 on IFEval to L's 69.7 and 30.3 on IFBench to 18.4, which matters for an assistant that has to respect a format or a word limit. The SFT models are tested in non-thinking mode only; there is no reasoning variant.
The phone figures are the paper's real contribution. At about the same INT4 weight memory as Meta's dense MobileLLM-Pro, MobileMoE-S is 1.8 to 3.8 times faster to the first token and 2.2 to 3.4 times faster at generation on the two flagship phones, with up to 22% less peak memory at 8K context. The quantised S keeps an average of 43.9 against MobileLLM-Pro's 45.5, so the small model trades a little accuracy for a lot of speed, and M at 52.7 beats it outright.
Shipping it is another matter. The weights are under the FAIR Noncommercial Research License, the architecture is not yet in upstream Transformers, so loading needs trust_remote_code, and the model cards document a PyTorch path rather than a packaged ExecuTorch export. Adoption reflects that: the most downloaded checkpoint, S-QAT, logged 209 downloads in the past month on Hugging Face. For a phone model that a business can actually ship, Gemma 4's smallest sizes under Apache 2.0 remain the practical comparison; MobileMoE is the design they may copy next.
Where it earns its place
An on-device ML researcher at a university lab
Compare a sparse and a dense model at the same memory budget on a real phone
Pair MobileMoE-S-QAT with MobileLLM-Pro, the dense baseline in Meta's paper, and profile both on a Snapdragon or Apple device. The paper's 1.8 to 3.8 times prefill gap gives a figure to reproduce, and the FAIR licence permits exactly this kind of research.
A mobile ML engineer at a phone maker's research group
Prototype an offline maths and homework helper before choosing a licensable model
MobileMoE-L-QAT, at 3.00 GB on disk, scores 73.2 on GSM8K after quantisation, against 4.1 for the INT4 MobileLLM-Pro in Meta's table. It can show what the feature could feel like, but anything shipped to customers needs a model with a commercial licence.
A graduate student working on mixture-of-experts kernels
Build and test expert-routing code for mobile CPUs
The checkpoints use a simple, documented layout: 60 experts, top-4 sigmoid routing, one shared expert and INT4 weights in groups of 32 with FP32 router weights. That makes them a clean target for a custom kernel, at no cost beyond the gated sign-up.
Before you commit
- Setup
- Accept the licence on Hugging Face, which asks for name, date of birth, country, affiliation and job title, then load with Transformers 4.57 or later and trust_remote_code=True. Meta recommends vLLM 0.10.2 or later with enforce_eager for batch evaluation.
- Integrations
- Hugging Face Transformers and vLLM on a server; Meta's paper runs the phones through ExecuTorch with XNNPACK and a custom fused MoE operator, and through MLX on the iPhone GPU. The model cards document only the PyTorch and Transformers path.
- Data and privacy
- Self-hosted, so prompts stay on your device or server. The FAIR Noncommercial Research License limits use to research and non-commercial purposes; the gate shares your Hugging Face username and email with Meta.
- Cost at scale
- No API price exists. On a phone the cost is battery and memory: INT4 checkpoints are 0.71, 1.59 and 3.00 GB, and runtime memory adds the KV cache on top. The BF16 L checkpoint is 10.65 GB, so a server test needs a GPU with about 12 GB or more.
- Limits
- English only, 8,192-token context, no image or audio input and no reasoning mode. Benchmark figures are Meta's own re-runs, and the smallest S model scores below MobileLLM-Pro on average after quantisation.
What’s good
- L averages 60.3 on 14 benchmarks against Qwen3.5 2B's 50.8 (Meta's re-run)
- Strong maths and code for its size: 76.8 GSM8K, 65.2 HumanEval
- INT4 checkpoints of 0.71, 1.59 and 3.00 GB fit current phone memory
- S runs 2.2 to 3.4 times faster at decode than MobileLLM-Pro on phones (Meta)
What’s not
- FAIR Noncommercial Research License rules out commercial products
- English only, text only, 8,192-token context
- Trails Qwen3.5 2B on instruction following (IFBench 18.4 to 30.3)
- Custom architecture needs trust_remote_code; no hosted provider
Best for
Researchers and on-device ML engineers studying how sparse models behave on phone hardware, where commercial use is not the goal.
MobileMoE (S, M, L) alternatives
MobileMoE (S, M, L): common questions
- Is MobileMoE (S, M, L) worth it?
- MobileMoE-L beats Qwen3.5 2B on most of Meta's table while running under a billion parameters per token, but the non-commercial licence keeps it out of any phone app you could sell. We rate it 3.6 out of 5.
- How much does MobileMoE (S, M, L) cost?
- Free to download after a gated sign-up, under the FAIR Noncommercial Research License: research and non-commercial use only. No hosted API or inference provider offers it.
- What are the downsides of MobileMoE (S, M, L)?
- FAIR Noncommercial Research License rules out commercial products; english only, text only, 8,192-token context; trails Qwen3.5 2B on instruction following (IFBench 18.4 to 30.3); custom architecture needs trust_remote_code; no hosted provider.
- Who is MobileMoE (S, M, L) best for?
- Researchers and on-device ML engineers studying how sparse models behave on phone hardware, where commercial use is not the goal.
- How much does MobileMoE (S, M, L) cost for a team or in production?
- No API price exists. On a phone the cost is battery and memory: INT4 checkpoints are 0.71, 1.59 and 3.00 GB, and runtime memory adds the KV cache on top. The BF16 L checkpoint is 10.65 GB, so a server test needs a GPU with about 12 GB or more.
- Is MobileMoE (S, M, L) safe for company data?
- Self-hosted, so prompts stay on your device or server. The FAIR Noncommercial Research License limits use to research and non-commercial purposes; the gate shares your Hugging Face username and email with Meta.
- What are the best MobileMoE (S, M, L) alternatives?
- The closest alternatives we have reviewed are Gemma 4, MiniCPM5-2B, SmolLM3-3B and LFM3.
Answers checked Nov 9, 2026.