Understand this story
Google Research releases the SLEEP offline memory consolidation method. According to a report by IT Home · AI, Google's research team has released a continual learning study called the "Sleep Framework," which divides model operation into two phases, awake and sleep, to address the problem of large models having fixed knowledge and difficulty absorbing new information; during the sleep phase, short-term context is written into long-term parameters through parameter expansion, mixture-of-experts memory blocks, and knowledge distillation, and synthetic samples generated through "dreaming" are used for reinforcement learning. Experiments show that the framework can handle contexts of up to about 10 million tokens, that performance degradation is close to zero in multi-stage continual learning on unseen languages, and that it achieves higher accuracy on math benchmarks such as AIME 24/25. Phoenix Technology reported that the study was made public on October 27, and that the paper is titled "Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories." There are currently no subsequent developments.
AI-generated from public reports
Developments
1 confirmed developments
Google Research releases SLEEP offline memory consolidation method
Source:IT Home · AI
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 30
IT Home · Artificial Intelligence
Google research team proposes Sleep Framework, enabling large models to learn while in use without forgetting old knowledge
The Google research team has released a continual learning study called the "Sleep Framework," which divides model operation into two phases—awake and sleep—to address the problem of large models' rigid knowledge and difficulty in absorbing new information. The sleep phase writes short-term context into long-term parameters through parameter expansion, mixture-of-experts memory blocks, and knowledge distillation, and generates synthetic samples via "dreaming" for reinforcement learning. Experiments show that the framework can handle contexts of up to about 10 million tokens, that performance degradation is close to zero in multi-stage continual learning on unseen languages, and that it achieves higher accuracy on mathematical benchmarks such as AIME 24/25.
Phoenix Tech
Google proposes a sleep framework that lets large models learn while in use without forgetting old knowledge
On October 27, Google's research team publicly released a continual learning study called the Sleep Framework, with the paper titled "Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories." The framework divides model operation into an awake phase and a sleep phase. During the sleep phase, short-term context is written into long-term parameters through parameter expansion, mixture-of-experts memory blocks, and knowledge distillation, and reasoning and generalization are improved through a dreaming step that generates synthetic samples. Experiments show that the model can handle contexts of up to about 10 million tokens, maintains near-zero performance degradation in multi-stage continual learning on unseen languages, and achieves higher accuracy on mathematical benchmarks such as AIME 24/25.
Oct 29
Shao Meng
Google Research releases SLEEP: using offline sleep cycles to mitigate catastrophic forgetting in large models
Google Research releases the SLEEP method, which lets LLMs periodically switch between wakeful interaction and sleep consolidation, writing short-term contextual information into scalable long-term parameters, and self-improving with synthetic data during the dreaming phase. The sleep phase unlocks reserved MoE memory blocks, combining generalized knowledge distillation and imitation learning to complete memory consolidation, then resets high-frequency temporary parameters. In evaluations of continual translation on unseen languages, SLEEP shows near-zero performance drop; on BABILong it is claimed to scale to about 10 million tokens; on AIME 24/25 and HMMT 25 it achieves approximately 79.2%, 69.0%, and 46.1%, respectively.
Heat trend for this story
- Current
- 8
- Period peak
- 14 Oct 30 16:00
- 24-hour change
- —
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.