arXiv:2603.03379cs.IRcs.AI2026-03被引 5

用小模型预判任务需求,高效检索大模型记忆,提升长时任务表现。

MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning

  • 用小型代理模型先推理任务目标,再精准调取记忆
  • 在8个基准上达成或超越当前最佳的检索准确率与任务完成率
  • 适合需要长期记忆的复杂推理任务研究者使用

随着大语言模型(LLMs)在长时间任务中的应用日益广泛,如何维持有效长时记忆成为关键挑战。现有方法常在成本与准确性间权衡:简单存储易丢失相关性,复杂索引(如记忆图)计算开销大且可能造成信息损失。依赖主模型处理所有记忆也效率低下。为此,我们提出 MemSifter,一种将记忆检索过程卸载至小型代理模型的新框架。该框架不增加主模型负担,而是由小模型先对任务进行推理,再决定所需信息。该方法索引阶段无需重计算,推理阶段开销极低。为优化代理模型,我们设计了基于任务结果的强化学习训练范式,奖励机制依据主模型实际任务表现,通过多轮交互评估检索记忆的实际贡献,并按递减贡献区分排序。结合课程学习与模型融合技术进一步提升性能。我们在包括 Deep Research 在内的八个 LLM 记忆基准上进行了评估,结果表明,该方法在检索准确率和最终任务完成率上均达到或超越现有最先进水平。我们已开源模型权重、代码与训练数据,以支持后续研究。

原文摘要 · Abstract (English)

As Large Language Models (LLMs) are increasingly used for long-duration tasks, maintaining effective long-term memory has become a critical challenge. Current methods often face a trade-off between cost and accuracy. Simple storage methods often fail to retrieve relevant information, while complex indexing methods (such as memory graphs) require heavy computation and can cause information loss. Furthermore, relying on the working LLM to process all memories is computationally expensive and slow. To address these limitations, we propose MemSifter, a novel framework that offloads the memory retrieval process to a small-scale proxy model. Instead of increasing the burden on the primary working LLM, MemSifter uses a smaller model to reason about the task before retrieving the necessary information. This approach requires no heavy computation during the indexing phase and adds minimal overhead during inference. To optimize the proxy model, we introduce a memory-specific Reinforcement Learning (RL) training paradigm. We design a task-outcome-oriented reward based on the working LLM's actual performance in completing the task. The reward measures the actual contribution of retrieved memories by mutiple interactions with the working LLM, and discriminates retrieved rankings by stepped decreasing contributions. Additionally, we employ training techniques such as Curriculum Learning and Model Merging to improve performance. We evaluated MemSifter on eight LLM memory benchmarks, including Deep Research tasks. The results demonstrate that our method meets or exceeds the performance of existing state-of-the-art approaches in both retrieval accuracy and final task completion. MemSifter offers an efficient and scalable solution for long-term LLM memory. We have open-sourced the model weights, code, and training data to support further research.

长时记忆代理模型强化学习LLM优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。