Google research team proposes Sleep Framework, enabling large models to learn while in use without forgetting old knowledge

The Google research team has released a continual learning study called the "Sleep Framework," which divides model operation into two phases—awake and sleep—to address the problem of large models' rigid knowledge and difficulty in absorbing new information. The sleep phase writes short-term context into long-term parameters through parameter expansion, mixture-of-experts memory blocks, and knowledge distillation, and generates synthetic samples via "dreaming" for reinforcement learning. Experiments show that the framework can handle contexts of up to about 10 million tokens, that performance degradation is close to zero in multi-stage continual learning on unseen languages, and that it achieves higher accuracy on mathematical benchmarks such as AIME 24/25.