连续更新的抽象记忆会失效,保留原始经验更可靠。
Useful Memories Become Faulty When Continuously Updated by LLMs

- 用大模型不断重写记忆,反而导致错误积累。
- 在ARC-AGI任务中,有54%的正确解因记忆更新而失败。
- 适合构建稳定智能体系统的研究人员参考。
学习过往经验依赖两种互补的记忆形式:情景痕迹(原始事件轨迹)和跨多轮提炼出的抽象知识。当前的智能体记忆系统倾向于后者:大模型将历史轨迹转化为文本记忆库并持续更新,实现无需参数调整的自我改进。然而我们发现,即使基于有用经验,这类抽象记忆也会随时间退化。记忆效用先上升后下降,甚至低于无记忆基准。更意外的是,即便从真实解出发,GPT-5.4在先前能解决的问题上仍有54%因记忆更新而失败。问题根源在于整合过程而非原始经验:相同轨迹在不同更新策略下产生截然不同的记忆;仅保留原始轨迹的对照组表现仍优于多个整合器。在控制环境ARC-AGI Stream中,自动保留原始轨迹的代理准确率是强制整合者的两倍;完全禁用整合(仅存景式管理)效果相当。实践上,可靠的记忆应将原始轨迹视为第一类证据,并显式控制整合行为,而非每次交互后自动触发。未来可靠的智能体记忆需要大模型在不覆盖依赖证据的前提下完成整合。
原文摘要 · Abstract (English)
Learning from past experience benefits from two complementary forms of memory: episodic traces -- raw trajectories of what happened -- and consolidated abstractions distilled across many episodes into reusable, schema-like lessons. Recent agentic-memory systems pursue the consolidated form: an LLM rewrites past trajectories into a textual memory bank that it continuously updates with new interactions, promising self-improving agents without parameter updates. Yet we find that such consolidated memories produced by today's LLMs are often faulty even when derived from useful experiences. As consolidation proceeds, memory utility first rises, then degrades, and can fall below the no-memory baseline. More surprisingly, even when consolidating from ground-truth solutions, GPT-5.4 fails on 54% of a set of ARC-AGI problems it had previously solved without memory. We trace the regression to the consolidation step rather than the underlying experience: the same trajectories yield qualitatively different memories under different update schedules, and an episodic-only control that simply retains those trajectories remains competitive with the consolidators we test. In a controlled ARC-AGI Stream environment that exposes Retain, Delete, and Consolidate actions, agents preserve raw episodes by default and double the accuracy of their forced-consolidation counterparts; disabling consolidation entirely (episodic management only) matches this auto regime. Practically, robust agent memory should treat raw episodes as first-class evidence and gate consolidation explicitly rather than firing it after every interaction. Looking forward, reliable agentic memory will require LLMs that can consolidate without overwriting the evidence they depend on.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。