Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Memorilla

Stanford research code that compresses a whole document collection or chat history into 16 memory tokens for a frozen LLM.

Reviewed by ReviewRamble editors · Updated

View on GitHub Open source

Share this page:XLinkedInFacebookRedditWhatsAppEmail

Our rating

3.8/5

Memorilla is a clever answer to long-term memory, beating retrieval methods with a fraction of the tokens, but for now it is research code that needs an 80 GB GPU.

What is Memorilla?

Memorilla is a memory system from Stanford's SNAP group. Instead of retrieving chunks of text, it reads an entire collection, such as a user's past chats, a long book or a document set, and hands a frozen language model a fixed 16 memory tokens shaped by the question being asked.

The paper reports the best average results against memory methods such as Mem0 and MemoRAG, and on FactKG it beats giving the model every document, 84.33% against 80.43%, while using 108 times fewer tokens. It also lifted an agent's success rate on TextWorld games from 18.6% to 24.2%. The repository has training and evaluation code and a released model for Qwen3-8B, under Apache 2.0.

Is Memorilla worth it?

Memorilla attacks the memory problem from a different side to retrieval. Rather than fetching the most similar chunks, it reads everything, a user's past conversations, a book, a document set, and condenses what matters for the current question into 16 memory tokens that a frozen model receives alongside the prompt.

The example in the README makes the case. A user had 107 earlier conversations, about 32,000 tokens; asked for breakfast ideas, Memorilla's Qwen3-8B tailored them to a family history of type 2 diabetes mentioned once in a single earlier chat, while plain Qwen3-8B gave a generic list. On benchmarks, the paper reports the best average against Mem0 and MemoRAG, and on FactKG it is more accurate than giving the model every document, 84.33% against 80.43%, with 108 times fewer tokens.

It is not a library to drop into a product yet. The code targets CUDA GPUs and was run on H100 80GB cards; the quick start needs one 80 GB GPU and downloads about 25 GB of models. The released memory module is trained for Qwen3-8B, so other models need training of their own, and the arXiv paper has not been peer reviewed.

Where it earns its place

  • An ML engineer building a personal assistant

    Test whether compressed memory beats retrieval on real chat histories

    Run the PersonaMem-v2 example, then train on your own data with the documented configs; budget an 80 GB GPU.

  • A research group working on agents

    Let an agent learn from its own past episodes

    Start from the TextWorld reinforcement learning setup, where Memorilla raised success from 18.6% to 24.2%.

Before you commit

Setup
Clone, create a Python 3.11 environment with uv, pip install -e ., optionally FlashAttention-2; the first run downloads about 25 GB of models.
Integrations
vLLM through a MemoryVLLM wrapper, Qwen3-8B, optional Weights & Biases, TextWorld.
Data and privacy
Runs on your own GPUs; Apache 2.0.
Cost at scale
Free code; training and evaluation were run on H100 80GB GPUs.
Limits
CUDA GPUs only; released module tied to Qwen3-8B; research-grade code.

What’s good

  • Fixed 16 memory tokens regardless of collection size
  • Beats Mem0 and MemoRAG on average, per the paper
  • 108 times fewer tokens than full context on FactKG
  • Training and evaluation code plus a released model

What’s not

  • Needs an 80 GB GPU for the quick start
  • Released module works with Qwen3-8B only
  • Research code, not a production library
  • Preprint, not peer reviewed

Best for

Researchers and ML engineers exploring long-term memory for assistants and agents who have large GPUs to experiment with.

Memorilla: common questions

Is Memorilla worth it?
Memorilla is a clever answer to long-term memory, beating retrieval methods with a fraction of the tokens, but for now it is research code that needs an 80 GB GPU. We rate it 3.8 out of 5.
How much does Memorilla cost?
Free and open source (Apache 2.0).
What are the downsides of Memorilla?
Needs an 80 GB GPU for the quick start; released module works with Qwen3-8B only; research code, not a production library; preprint, not peer reviewed.
Who is Memorilla best for?
Researchers and ML engineers exploring long-term memory for assistants and agents who have large GPUs to experiment with.
How much does Memorilla cost for a team or in production?
Free code; training and evaluation were run on H100 80GB GPUs.
Is Memorilla safe for company data?
Runs on your own GPUs; Apache 2.0.

Answers checked Nov 15, 2026.