让大模型学会精简记忆，多轮对话不膨胀
MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning
- 用强化学习训练模型只保留关键信息，动态管理记忆
- 多轮对话中上下文长度几乎不变，节省计算资源
- 适合需要长期交互的智能助手、搜索系统
基于大模型的搜索代理常将完整交互历史拼接进上下文，导致输入过长、噪声多，增加计算成本和显存占用。为解决此问题，我们提出 MemSearcher 框架，在多轮交互中维护紧凑记忆，仅保留与问题相关的信息，从而保持上下文长度稳定。训练挑战在于每条轨迹跨越多轮且处于不同大模型上下文中，使每轮成为独立优化目标。为此，我们引入多上下文 GRPO，将轨迹级优势传播至所有回合，实现端到端优化。实验表明，MemSearcher 在多个公开数据集上优于强基线（如 ReAct 风格的历史拼接方法），同时在多轮交互中维持近乎恒定的词元数量。代码与模型将开源于 https://github.com/icip-cas/MemSearcher。
原文摘要 · Abstract (English)
LLM-based search agents often concatenate the full interaction history into the context, producing long and noisy inputs, and increasing compute cost and GPU memory overhead. To address this issue, we propose MemSearcher, an agent framework that maintains a compact memory during multi-turn interactions, retaining only question-relevant information and thereby keeping the context length stable across turns. Training MemSearcher is challenging because each trajectory spans multiple turns under different LLM contexts, making each turn an independent optimization target in reinforcement learning. We introduce multi-context GRPO, which propagates trajectory-level advantages to all turns for end-to-end optimization. Experiments demonstrate that MemSearcher outperforms strong history-concatenation (ReAct-style) baselines on a range of public datasets while maintaining nearly constant token counts across multi-turn interactions. The code and models will be publicly available at https://github.com/icip-cas/MemSearcher
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。