arXiv:2602.12108cs.AI2026-02被引 6

让大模型自己管理记忆,突破上下文长度限制。

The Pensieve Paradigm: Stateful Language Models Mastering Their Own Context

  • 引入内部推理循环,让模型主动控制记忆工具。
  • 长文档问答准确率超越标准模型,研究任务达52%正确率。
  • 适合需要长期记忆与复杂推理的AI应用开发者。

在《哈利·波特》中,邓布利多将记忆存入冥想盆以备后用。而在人工智能领域,尽管已有成熟的数据库和检索系统,模型却仍缺乏操作这些记忆的‘魔杖’,只能被动接受人工设计的上下文。本文提出StateLM,一种具备内部推理循环的基础模型,使其能自主管理自身状态。我们为模型配备上下文修剪、文档索引、笔记记录等记忆工具,并训练其主动运用这些工具。通过学习动态构建上下文,模型突破了固定窗口的架构束缚。实验表明,在不同规模模型上,StateLM在长文档问答任务中持续领先标准LLM;在聊天记忆任务中,准确率提升10%至20%;在深度研究任务BrowseComp-Plus中,表现差距显著:StateLM达到52%准确率,而标准模型仅约5%。该方法使大模型从被动预测者转变为具有状态感知能力的智能体,使推理成为可管理的状态过程。

原文摘要 · Abstract (English)

In the world of Harry Potter, when Dumbledore's mind is overburdened, he extracts memories into a Pensieve to be revisited later. In the world of AI, while we possess the Pensieve-mature databases and retrieval systems, our models inexplicably lack the "wand" to operate it. They remain like a Dumbledore without agency, passively accepting a manually engineered context as their entire memory. This work finally places the wand in the model's hand. We introduce StateLM, a new class of foundation models endowed with an internal reasoning loop to manage their own state. We equip our model with a suite of memory tools, such as context pruning, document indexing, and note-taking, and train it to actively manage these tools. By learning to dynamically engineering its own context, our model breaks free from the architectural prison of a fixed window. Experiments across various model sizes demonstrate StateLM's effectiveness across diverse scenarios. On long-document QA tasks, StateLMs consistently outperform standard LLMs across all model scales; on the chat memory task, they achieve absolute accuracy improvements of 10% to 20% over standard LLMs. On the deep research task BrowseComp-Plus, the performance gap becomes even more pronounced: StateLM achieves up to 52% accuracy, whereas standard LLM counterparts struggle around 5%. Ultimately, our approach shifts LLMs from passive predictors to state-aware agents where reasoning becomes a stateful and manageable process.

状态建模记忆管理长序列推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。