提出新基准InMind，揭示智能体记忆的隐性关联盲区。
Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory
- 构建125项任务的专家验证基准，区分记忆缺失、知识不足与未激活三类原因。
- 基础模型在间接查询中正确率达84.0%，主流记忆系统最高仅14.4%。
- 关键问题在于查询条件下的记忆路由机制，适合评估记忆可及性研究者使用。
长期记忆系统将用户输入存储于外部并按相关查询召回。该机制依赖一个未明言却自然的前提：所需记忆应与查询有可见关联。但世界知识会打破此前提。例如坚果过敏需影响马卡龙回复（因含杏仁粉），而两文本无共享线索供检索器识别。我们称此为隐性关联盲区，并提出InMind——一个包含125项任务、覆盖十类生活场景、113项任务基于可引用公共来源的专家验证基准。其配对控制分离了三种混淆解释：事实未存储、模型缺乏桥梁知识或事实已存但未浮现。结果清晰：当记忆置于上下文时，基础模型可正确回答84.0%的间接查询；而六种向量、图谱与代理式记忆系统在相同记忆召回率高达100%的情况下，间接查询准确率最高仅14.4%。嵌入维度提升八倍虽提高所有系统的目标召回率，但差距基本不变。一个简单诊断探针若在查询前保持记忆可见，即可恢复大部分性能差距，表明失败根源在于查询条件下的接口设计，指向‘路由’这一开放问题。InMind正是为此类评估而建。
原文摘要 · Abstract (English)
Long-term memory systems store what a user says in an external store and retrieve it when a related query arrives. This interface rests on an assumption so natural that it is rarely stated: a memory that is needed will resemble the query that needs it. World knowledge breaks the assumption. A tree-nut allergy should change the answer to a macaron request through their almond-flour ingredient, yet the two texts share no cue a retriever can see. We call this failure mode the implicit-association blind spot and introduce InMind, a 125-task, expert-verified benchmark spanning ten life domains, with 113 tasks grounded in citable public sources. Its paired controls separate three explanations that existing evaluations conflate: the fact was never stored, the model lacks the bridging knowledge, or the fact was stored and never surfaced. The verdict is clean. With the decisive memory placed in context, the backbone answers 84.0 percent of indirect queries; when the same memory must be retrieved, six vector, graph, and agentic memory systems reach at most 14.4 percent, even though they recall the same facts on demand at up to 100 percent. An embedding with eight times the dimensionality raises answer-blind target recall for every system yet leaves the gap essentially intact. A minimal diagnostic probe that keeps memory visible before the query arrives recovers most of the gap, locating the failure in the query-conditioned interface itself and pointing to routing, deciding which facts must stay visible, as the open problem InMind is built to score.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。