A language model stops learning the day it ships. Anything it meets afterwards lives only in its context window and vanishes when the session closes, and retraining on new data tends to erase what it knew before, a problem known as catastrophic forgetting. Google Research has published a framework that tries to fix both at once by giving the model a routine of waking and sleeping.
During the wake phase the model works normally and holds new material in fast, short-term memory. During sleep it stops taking inputs and writes that material into its weights. The authors, Ali Behrouz, Farnoosh Hashemi, Adel Javanmard and Vahab Mirrokni, do this by unlocking spare low-rank expert blocks in a mixture-of-experts design built on their Hope architecture, so new knowledge lands in fresh capacity rather than on top of old parameters. Transfer uses on-policy distillation plus an imitation reward based on semantic similarity and Levenshtein distance, and the temporary memory is then cleared. A separate dreaming stage has the router switch on unusual experts to generate synthetic practice examples, keeps the most useful ones by gradient-based selection and fine-tunes on them.
The reported results are broad. On translation into Manchu and Kalamang, languages the model had not seen, performance barely dropped across stages while in-context learning, Cartridges and plain fine-tuning forgot heavily. On BABILong the method held up to 10M tokens of context. It scored 48.9% on single-task SQuAD knowledge injection and 46.2% under continued pretraining, ahead of SEAL, and on maths it reached 79.2% on AIME 24, 69.0% on AIME 25 and 46.1% on HMMT 25, beating GRPO and on-policy self-distillation baselines.
The idea sits in a line of Google work on memory, including Titans, which the post names as a baseline. It matters for anyone building assistants or agents that should remember a customer or a codebase over months; today that job falls to retrieval and long contexts, which reset or grow expensive.
What the post leaves out is what a practitioner would need first. It does not name the base model, report how much compute a sleep cycle uses or how long it takes, or say how often a deployed model would need to sleep. The comparisons are against the authors' chosen baselines, and the borrowing from neuroscience is metaphor, as they say themselves. Until a paper with those details and outside replication appears, treat the numbers as a promising research result.