大模型代理用记忆存储经验，但记忆竞争让持续学习难题转移至记忆访问环节。
When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agents
- 分离记忆表征与组织方式，构建(k,v)分析框架
- 抽象过程记忆比详细轨迹更易迁移，难任务受负迁移影响更大
- 细粒度记忆组织未必更好，可能引发严重遗忘
带有外部记忆的大模型代理为持续学习提供了看似便捷的路径：无需更新模型参数，仅将经验积累于外部记忆中，从而避开参数化学习中的稳定性-可塑性困境。然而我们发现，这一挑战并未消失，而是转移到了记忆层面。在有限上下文窗口下，新旧经验在检索时产生竞争，使持续学习的瓶颈从参数更新转移到记忆访问。为此，我们提出一个(k,v)框架，解耦外部记忆的两个核心设计维度：经验如何表征，以及如何组织以支持检索。在ALFWorld和BabyAI上的序列任务实验表明，抽象的过程记忆比详细轨迹具有更强的迁移能力，而负迁移对困难任务的损害尤为显著。此外，更细粒度的记忆组织并非总是有益：某些设计虽能促进正向迁移，却同时导致严重遗忘。这些结果表明，外部记忆并未解决持续学习问题，而是将其重塑为记忆表征与检索设计的新挑战。
原文摘要 · Abstract (English)
Memory-augmented LLM agents offer an appealing shortcut to continual learning: rather than updating model parameters, they accumulate experience in external memory, seemingly sidestepping the stability-plasticity dilemma of parametric learning. We show that this challenge does not disappear but resurfaces at the memory level. Under a limited context window, old and new experiences compete during retrieval, relocating the continual-learning bottleneck from parameter updates to memory access. To study this phenomenon, we introduce a (k,v) framework that disentangles two fundamental design axes of external memory: how experience is represented and how it is organized for retrieval. Across sequential-task experiments in ALFWorld and BabyAI, we find that abstract procedural memories transfer more reliably than detailed trajectories, while negative transfer disproportionately harms the hard cases. Moreover, finer-grained memory organization is not universally beneficial: designs that yield strong forward transfer can simultaneously induce severe forgetting. Together, these results reveal that external memory does not resolve the continual-learning problem; it reshapes it into a problem of memory representation and retrieval design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。