让语言智能体自动整理记忆,提升长期学习能力。
Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents

- 分离快速记录与缓慢整理,离线重构记忆以发现共性模式
- 在ScienceWorld上比基线高7分,内存仅用1/12
- 无需重训即可跨场景通用,适合需要持续学习的智能体
语言智能体在连续任务流中运行时,现有记忆系统难以将积累经验转化为可复用知识。检索增强与结构化记忆方法虽能有效记录会话内容,但通常将获取与整合耦合于在线过程,使智能体缺乏跨会话全局视角,无法发现重复模式、抽象通用流程或清除冗余条目。受互补学习系统理论启发,我们提出Auto-Dreamer,一种用于语言智能体记忆的可学习离线整合器。该方法将快速会话级记忆获取与慢速跨会话整合解耦。给定一个类型化的记忆池工作区,整合器将其视为只读证据,通过有限工具调用检查条目及其来源轨迹,合成一组紧凑的新替换集,实现跨会话抽象并取代原区域。采用GRPO训练,以端到端智能体性能为奖励信号,学习如何从快速在线经验中整合记忆。仅在ScienceWorld轨迹上训练后,Auto-Dreamer在ScienceWorld上比固定、强化学习训练和提示型记忆基线高出7分,且主动记忆池大小仅为最强基线的1/12;在未见的ALFWorld和WebArena上仍保持领先,且在ALFWorld上内存使用量仅为最强基线的1/6。
原文摘要 · Abstract (English)
Language agents increasingly operate over streams of related tasks, yet existing memory systems struggle to convert accumulated experience into reusable knowledge. Retrieval-augmented and structured memory methods record per-session observations effectively, but often couple acquisition and consolidation into a single online process, leaving the agent without a global view across sessions to discover recurring patterns, abstract shared procedures, or prune redundant entries. Inspired by complementary learning systems theory, we propose Auto-Dreamer, a learned offline consolidator for language-agent memory. Auto-Dreamer decouples fast per-session memory acquisition from slow cross-session consolidation. Given a selected working region of a typed memory bank, the consolidator treats the region as read-only evidence, performs bounded tool-use to inspect entries and provenance-linked source trajectories, and synthesizes a fresh compact replacement set that abstracts across sessions and supersedes the original region. We train Auto-Dreamer via GRPO, using end-to-end agent performance as the reward signal to learn how to consolidate memories acquired through fast online experience. Trained on ScienceWorld trajectories alone, Auto-Dreamer outperforms fixed, RL-trained, and prompted memory baselines on ScienceWorld by 7 points while using an active memory bank 12$\times$ smaller than the strongest baseline, and continues to lead on held-out ALFWorld and WebArena without retraining -- using 6$\times$ less memory than the strongest baseline on ALFWorld.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。