arXiv:2608.28476cs.CL2026-08中稿 · EMNLP

让AI智能管理记忆,高效处理长流程任务。

ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

论文配图:ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
图 1 · 摘自论文原文
  • 用规划、长期记忆和动态压缩工具扩展上下文管理能力。
  • 通过熵变识别关键决策点,提升动作优势评估精度。
  • 在问答与深度搜索任务中表现更优且内存更少。

长周期智能体任务要求大语言模型在多轮交互中迭代地检索、整合并维护分散信息,但保留全部交互历史会导致工作上下文持续膨胀。现有主动上下文管理方法虽允许模型使用专用工具编辑自身上下文,仍存在三大局限:(1) 工具集受限于搜索、删除和摘要,缺乏全局规划、长期记忆和自适应压缩能力;(2) 探索效率低,未区分不同操作对最终结果的异质影响;(3) 信用分配粗粒度,将轨迹级奖励均摊给所有中间编辑动作。为此,我们提出 ContextPilot,一种面向长周期智能推理的主动上下文管理框架。系统性扩充工具集,引入规划、长期记忆和软上下文卸载工具。进一步设计适配上下文管理的强化学习方法,利用上下文与熵的变化识别关键编辑决策以进行分支采样,并从通过该动作的所有分支轨迹中估计动作级优势。在长上下文问答与深度搜索任务上的实验表明,ContextPilot 在更紧凑的工作上下文中实现更强性能,显著优于多种基线模型与基准。代码已开源:https://github.com/Tencent/ContextPilot。

原文摘要 · Abstract (English)

Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, a proactive context management framework for long-horizon agentic reasoning. Our approach systematically augments the toolset with planning, long-term memory, and soft context offloading tools. We further propose an RL method tailored for context management, which uses context and entropy variation to identify critical editing decisions for branch sampling and estimates action-level advantages from all branched trajectories that pass through the corresponding context editing action. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming existing baselines across various base models and benchmarks. Code is available at https://github.com/Tencent/ContextPilot.

上下文管理强化学习大模型推理智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。