What is Memorilla?
Memorilla is a memory system from Stanford's SNAP group. Instead of retrieving chunks of text, it reads an entire collection, such as a user's past chats, a long book or a document set, and hands a frozen language model a fixed 16 memory tokens shaped by the question being asked.
The paper reports the best average results against memory methods such as Mem0 and MemoRAG, and on FactKG it beats giving the model every document, 84.33% against 80.43%, while using 108 times fewer tokens. It also lifted an agent's success rate on TextWorld games from 18.6% to 24.2%. The repository has training and evaluation code and a released model for Qwen3-8B, under Apache 2.0.
Is Memorilla worth it?
Memorilla attacks the memory problem from a different side to retrieval. Rather than fetching the most similar chunks, it reads everything, a user's past conversations, a book, a document set, and condenses what matters for the current question into 16 memory tokens that a frozen model receives alongside the prompt.
The example in the README makes the case. A user had 107 earlier conversations, about 32,000 tokens; asked for breakfast ideas, Memorilla's Qwen3-8B tailored them to a family history of type 2 diabetes mentioned once in a single earlier chat, while plain Qwen3-8B gave a generic list. On benchmarks, the paper reports the best average against Mem0 and MemoRAG, and on FactKG it is more accurate than giving the model every document, 84.33% against 80.43%, with 108 times fewer tokens.
It is not a library to drop into a product yet. The code targets CUDA GPUs and was run on H100 80GB cards; the quick start needs one 80 GB GPU and downloads about 25 GB of models. The released memory module is trained for Qwen3-8B, so other models need training of their own, and the arXiv paper has not been peer reviewed.
Where it earns its place
An ML engineer building a personal assistant
Test whether compressed memory beats retrieval on real chat histories
Run the PersonaMem-v2 example, then train on your own data with the documented configs; budget an 80 GB GPU.
A research group working on agents
Let an agent learn from its own past episodes
Start from the TextWorld reinforcement learning setup, where Memorilla raised success from 18.6% to 24.2%.
Before you commit
- Setup
- Clone, create a Python 3.11 environment with uv, pip install -e ., optionally FlashAttention-2; the first run downloads about 25 GB of models.
- Integrations
- vLLM through a MemoryVLLM wrapper, Qwen3-8B, optional Weights & Biases, TextWorld.
- Data and privacy
- Runs on your own GPUs; Apache 2.0.
- Cost at scale
- Free code; training and evaluation were run on H100 80GB GPUs.
- Limits
- CUDA GPUs only; released module tied to Qwen3-8B; research-grade code.
What’s good
- Fixed 16 memory tokens regardless of collection size
- Beats Mem0 and MemoRAG on average, per the paper
- 108 times fewer tokens than full context on FactKG
- Training and evaluation code plus a released model
What’s not
- Needs an 80 GB GPU for the quick start
- Released module works with Qwen3-8B only
- Research code, not a production library
- Preprint, not peer reviewed
Best for
Researchers and ML engineers exploring long-term memory for assistants and agents who have large GPUs to experiment with.
Memorilla: common questions
- Is Memorilla worth it?
- Memorilla is a clever answer to long-term memory, beating retrieval methods with a fraction of the tokens, but for now it is research code that needs an 80 GB GPU. We rate it 3.8 out of 5.
- How much does Memorilla cost?
- Free and open source (Apache 2.0).
- What are the downsides of Memorilla?
- Needs an 80 GB GPU for the quick start; released module works with Qwen3-8B only; research code, not a production library; preprint, not peer reviewed.
- Who is Memorilla best for?
- Researchers and ML engineers exploring long-term memory for assistants and agents who have large GPUs to experiment with.
- How much does Memorilla cost for a team or in production?
- Free code; training and evaluation were run on H100 80GB GPUs.
- Is Memorilla safe for company data?
- Runs on your own GPUs; Apache 2.0.
Answers checked Nov 15, 2026.