arXiv:2508.04664cs.CLcs.AI2025-08被引 10

让大模型像人一样主动管理记忆,提升长文本理解能力

Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management

  • 用分段、摘要、搜索等工具主动控制内部记忆
  • 在多个长文本任务上性能显著提升,无需额外训练
  • 适合需要精准推理的长序列场景,如法律分析

大语言模型在处理长上下文时因前摄干扰(proactive interference)导致性能下降,即早期无关信息会干扰后续推理与记忆。现有研究多依赖外部记忆系统,我们提出互补方案:赋予大模型主动上下文管理(ACM)能力。提出Sculptor框架,包含三类工具:(1) 上下文分段,(2) 摘要、隐藏与恢复,(3) 精确搜索。该方法使模型能主动调控注意力与工作记忆,类似人类筛选信息、忽略干扰。在多个长上下文基准测试中,Sculptor在不需特定训练的情况下显著提升性能,利用模型固有的工具调用与指令遵循能力。为进一步优化策略,引入动态上下文感知强化学习(RL)方法,训练智能体主动修改对话历史。Sculptor不仅缓解前摄干扰,还为多样化长文本任务提供可靠推理基础,证明显式上下文控制比单纯扩大上下文窗口更关键。

原文摘要 · Abstract (English)

Large Language Models (LLMs) suffer from significant performance degradation when processing long contexts due to proactive interference, where irrelevant information in earlier parts of the context disrupts reasoning and memory recall. While most research focuses on external memory systems to augment LLMs' capabilities, we propose a complementary approach: empowering LLMs with Active Context Management (ACM) tools to actively sculpt their internal working memory. We introduce Sculptor, a framework that equips LLMs with three categories of tools: (1) context fragmentation, (2) summary, hide, and restore, and (3) precise search. Our approach enables LLMs to proactively manage their attention and working memory, analogous to how humans selectively focus on relevant information while filtering out distractions. Experimental evaluation on diverse long-context benchmarks demonstrates that Sculptor significantly improves performance even without specific training, leveraging LLMs' inherent tool-calling and instruction-following capabilities. To further optimize these strategies, we introduce a novel dynamic context-aware reinforcement learning (RL) approach, advancing the training of an agent that actively modifies its own conversational history. By enabling Active Context Management, Sculptor not only mitigates proactive interference but also provides a cognitive foundation for more reliable reasoning across diverse long-context tasks-highlighting that explicit context-control strategies, rather than merely larger token windows, are key to robustness at scale.

大模型上下文管理推理增强强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。