让智能体主动检索记忆，提升长期学习效率
Ask Only When Needed: Proactive Retrieval from Memory and Skills for Experience-Driven Lifelong Agents
- 设计主动检索机制，根据任务进展决定何时调用记忆
- 在多个场景中提升成功率32%，减少33%交互轮次
- 适合需要持续学习与自我优化的长期智能体系统
在线终身学习智能体不仅需决定如何行动，还需判断何时调用过往经验以持续改进长周期任务。现有方法多被动检索记忆，如任务初始化或每步后，因而错失交互中出现的知识缺口。我们提出 ProactAgent，一种基于结构化经验库的主动检索框架。ProactAgent 通过 ExpOnEvo 联合更新策略并优化记忆，将历史交互整理为事实、情节和技能三类存储库。同时引入 ProactRL，将检索视为显式策略动作，学习何时何地检索。通过对比相同交互前缀下有无检索的延续表现，提供细粒度过程奖励，仅在检索能提升任务结果或效率时才触发。在 SciWorld、AlfWorld 和 StuLife 上的实验表明，ProactAgent 持续优于所有基线，成功率达基线提升32%，交互轮次减少超33%。代码将公开于 GitHub。
原文摘要 · Abstract (English)
Online lifelong learning agents must decide not only how to act but also when to consult prior experience to continually improve on long-horizon tasks. Existing methods typically retrieve memories passively, such as at task initialization or after each step, and therefore miss knowledge gaps that arise during interaction. We propose ProactAgent, an experience-driven lifelong learning framework for proactive retrieval over a structured Experience Base. ProactAgent continually improves through ExpOnEvo, which jointly updates policies and refines memory, organizing past interactions into factual, episodic, and skill repositories. It further introduces ProactRL, which treats retrieval as an explicit policy action and learns when and what to retrieve. By comparing paired continuations from identical interaction prefixes with and without retrieval, ProactRL provides step-level process rewards that encourage retrieval only when it improves task outcomes or efficiency. Experiments on SciWorld, AlfWorld, and StuLife show that ProactAgent consistently outperforms all baselines, achieving up to 32% relative improvement in success rate and over 33% reduction in interaction rounds. Our code will be publicly available at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。