让大模型定期‘睡觉’，用离线循环提升长程推理能力
Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference
- 模型定期将近期上下文转为持久快速权重，清空缓存后‘睡觉’
- 睡眠时进行N次离线循环更新状态空间模型权重，显著提升复杂推理准确率
- 适合需要深度推理的长序列任务，如数学题和图谱检索
基于Transformer的大语言模型在长序列任务中应用日益广泛，但其注意力机制随上下文长度增长而性能下降。为此，我们研究了一种类睡眠的巩固机制：模型周期性地将近期上下文转换为持久的快速权重，并清空键值缓存。在‘睡眠’阶段，模型对积累的上下文执行N次离线递归遍历，通过学习到的局部规则更新状态空间模型（SSM）块中的快速权重。推理时，额外计算被转移到睡眠阶段，而唤醒时的预测延迟保持不变。我们在受控合成任务（如细胞自动机和多跳图检索）及真实数学推理任务上进行了测试，发现传统Transformer和SSM-注意力混合模型均失败。结果显示，增加睡眠时长N能提升模型性能，尤其在需深层推理的任务上收益最大。
原文摘要 · Abstract (English)
Transformer-based large language models are increasingly used for long-horizon tasks; however, their attention mechanism scales poorly with context length. To handle this, we study a sleep-like consolidation mechanism in which a model periodically converts recent context into persistent fast weights before clearing its key-value cache. During sleep, the model performs $N$ offline recurrent passes over the accumulated context and updates the fast weights in its state-space model (SSM) blocks through a learned local rule. During inference, this shifts extra computation to sleep while preserving the latency of wake-time prediction. We test our method on controlled synthetic tasks, including cellular automata and multi-hop graph retrieval, as well as a realistic math reasoning task, on which a regular transformer as well as SSM-attention hybrid models fail. We then show that increasing sleep duration $N$ for our models improves performance, with the largest gains on examples that require deeper reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。