arXiv:2609.08279cs.CLcs.AI2026-09

通过恢复实验区分记忆遗忘中可挽回与不可挽回的损失,揭示不同淘汰策略的真实影响。

What Eviction Destroys: A Restore-Counterfactual Audit of Forgetting in Agent Memory

  • 设计恢复反事实实验,逐题检验被删信息是否可复原
  • 80k预算下多数错误不可挽回(占比60%-73%),8k时达100%
  • 首次在标准对话数据集上实现对记忆淘汰的细粒度审计

智能体记忆系统在历史长度超过固定令牌预算时必须淘汰信息。现有预算-准确率边界仅衡量准确率损失,无法区分淘汰导致的不可逆损失与可恢复的检索失败。本文引入恢复反事实方法:对每道题,在读取时恢复其正确证据并重跑阅读器。通过比较恢复前后的正确性,以及证据是否在淘汰后仍保留,将每个可回答错误分类为可恢复、不可逆或残留(恢复后仍错)。在LongMemEval-S上,以GPT-4o-mini为读者和评判者,评估FIFO、随机、冗余感知及LLM重要性四种淘汰策略,在三种预算和两种检索方式下。在80k令牌预算下,顶K检索时,前三种策略不可逆错误占比为0.67–0.73,而LLM重要性为0.60;在8k预算时,所有策略均达1.00。可恢复错误仅出现在顶K检索中,强制注入正确答案则不存在,因此预算-准确率结果若不报告检索方式则不可比。匹配准确率分析未发现准确率相近策略间不可逆率差异(分辨率1.2–6个百分点),但能检测出故意破坏的对照组。据我们所知,这是首个针对外部智能体记忆库在标准对话基准上的逐项、逐题恢复反事实审计。

原文摘要 · Abstract (English)

Agent memory systems must discard stored information when their history exceeds a fixed token budget. Existing budget-accuracy frontiers quantify the resulting loss in accuracy, but do not distinguish irreversible losses caused by eviction from recoverable retrieval failures. We introduce the restore counterfactual, a per-question paired intervention that reinstates the question's gold evidence in the read-time context and reruns the same reader. Combining the change in correctness with whether the evidence was retained after eviction classifies each oracle-answerable error as recoverable, irreversible, or residual; in the residual case, the answer remains incorrect after restoration. We evaluate FIFO, random, redundancy-aware, and LLM-importance eviction on LongMemEval-S at three budgets and under two retrieval regimes, using GPT-4o-mini as the primary reader and judge and GPT-5.4-mini as a robustness reader. Under top-k retrieval at an 80k-token budget, the irreversible share among errors corrected by restoration is 0.67-0.73 for FIFO, random, and redundancy-aware eviction, compared with 0.60 for LLM-importance. At 8k tokens, it reaches 1.00 for all four policies. Recoverable errors occur under top-k retrieval at 80k tokens but are absent under forced-gold injection by construction, so budget-accuracy results are not directly comparable unless the retrieval regime is reported. An exploratory matched-accuracy analysis detects no difference in irreversible rate among accuracy-matched policy pairs at a resolution of 1.2-6 percentage points. The same analysis detects the deliberately destructive control. To our knowledge, this is the first per-item, per-question restore-counterfactual audit of eviction for external agent-memory stores on a standard conversational benchmark.

记忆淘汰反事实审计智能体记忆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。