发现大模型在对话中无法及时更新过时信息，提出可训练的环境来解决此问题。
Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents
- 构建可衡量并奖励当前事实使用的强化学习环境
- 小模型经训练后对过时信息识别准确率从9.0%升至16.7%
- 首次证明记忆更新失败可通过训练缓解，非仅模型规模问题
大型语言模型代理在长时间多轮对话中需处理动态变化的事实（如用户搬家、价格调整、计划修改），正确行动依赖于使用最新信息并丢弃已被覆盖的信息。我们在真实对话数据上隔离该能力，发现其为独立且未解决的缺陷：在LongMemEval的知识更新子集上，用有限自维护记忆替代完整上下文使准确率从92%降至77%，即使在前沿模型gpt-5.4上仍显著（配对McNemar检验，p<0.005），且随模型规模扩大而持续存在。问题在于记忆维护而非理解能力，且更强模型无法弥补。进一步实验表明，这并非内存不足：对话长度增长24倍时，准确率从68%降至28%，即便按比例增加内存也无改善（28%→28%，n=25）。准确率下降与会话长度相关，而非压缩比。我们发布Supersede，一个基于verifiers/prime-rl架构的开放强化学习环境，将事实时效性转化为训练信号：奖励当前值回答，惩罚陈旧回答。最终通过GRPO微调小模型Qwen2.5-3B，在真实未见对话上将其保持准确率从9.0%提升至16.7%（单次运行），且检查点曲线单调上升，表明策略学习有效。这是首个以时间事实时效性为目标的可训练环境，也是首个证明该差距可被训练缩小的证据。
原文摘要 · Abstract (English)
Large language model (LLM) agents operate over long, multi-session interactions in which facts change: a user moves, a price updates, a plan is revised. Acting correctly requires using the current value of a fact and discarding values that have been superseded. We isolate this ability on real conversational data and show that it is a distinct, unsolved failure. On the knowledge-update subset of LongMemEval, replacing an agent's full context with a bounded, self-maintained memory drops accuracy from 92% to 77% even on a frontier model (gpt-5.4), a gap that is statistically significant (paired McNemar p<0.005) and persists across model scale while full-context accuracy saturates near 92%. The bottleneck is therefore memory maintenance, not comprehension, and is not closed by a stronger model. We then ask whether this is merely an undersized memory, and find it is not: as the conversation grows 24x, accuracy falls further (from 68% to 28%), and granting the agent proportionally more memory yields no detectable recovery (28% to 28%, n=25). The failure scales with the length of the conversation, not the compression ratio. We release Supersede, an open reinforcement-learning environment (on the verifiers / prime-rl stack) that turns this measurement into a training signal: agents are rewarded for answering from the current value and penalized for stale ones. Finally, we close the loop and show the gap is trainable: GRPO fine-tuning a small open model (Qwen2.5-3B) on this environment nearly doubles its held-out supersession accuracy on real, unseen conversations (9.0% to 16.7%, a single run), along a monotonic checkpoint curve indicating the learned policy, not the harness, carries the gain. To our knowledge this is the first trainable environment whose reward targets temporal fact-currency, and the first evidence the supersession gap can be trained down, not only measured.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。