arXiv:2605.29630cs.CLcs.AI2026-05被引 1

提出实体碰撞协议,精准分离检索提升来源,揭示嵌入模型真实性能。

Entity-Collision: A Stratified Protocol for Attributing Retrieval Lift in Agent Memory

  • 构建基于实体碰撞的测试协议,强制干扰项共享答案实体词元。
  • 发现MiniLM-384在所有场景下表现最优,大模型不总更优。
  • 适合评估代理记忆系统中嵌入模型的真实检索能力。

端到端代理记忆基准测试通常仅报告单个命中率@k,混淆了词汇泄漏(查询/正确答案/干扰项实体重叠)与标签混合(偏好、服务、工具平均化)。我们提出实体碰撞协议,该系统无关协议通过构造使BM25下限固定——每个干扰项均共享答案的实体词元——并按判别器标签分层查询,使任何超过BM25的提升均可归因于嵌入器。在开源代理记忆测试平台中,对5个标签×3个嵌入器×5种碰撞程度进行配对自举95%置信区间分析,结果呈现双轴模式:256维哈希三元组仅在封闭词汇词法标签且高碰撞时有效;MiniLM-384在两轴上均占优;2.7倍参数量的BGE-large并未全面超越MiniLM——其在意图类查询中胜出,但在词法类查询中落后。编码器容量并非决定性瓶颈。合成意图标签空模型在LongMemEval(n=500)上重现为单会话偏好召回悬崖。LoCoMo上的自适应向量加权路由表现为可控零信号:11.7个百分点的理论最优余量存在,但所测信号无法恢复。全部26张结果表和37个复现脚本均已版本控制并通过公开注册库验证;协议在确定性治理的记忆测试平台(事件溯源决策日志、DAG状态机模式生命周期)上运行,确保每次报告的置信区间可从输入流逐字复现。

原文摘要 · Abstract (English)

End-to-end agent-memory benchmarks report a single hit@k per retriever, confounding lexical leakage (uncontrolled query/gold/distractor entity overlap) with tag-mixing (preferences, services, tools averaged together). We propose entity-collision, a system-agnostic protocol that pins the BM25 floor by construction -- every distractor shares the answer's entity tokens -- and stratifies queries by discriminator tag, so any lift over BM25 is attributable to the embedder. Applied to an open-source agent-memory testbed across 5 tags x 3 embedders x 5 collision degrees with paired-bootstrap 95% CIs, the protocol reveals a two-axis pattern: a 256-d hash trigram helps only on closed-vocabulary lexical tags at deep collision; MiniLM-384 dominates both axes; and a 2.7x-parameter BGE-large does not uniformly improve on MiniLM -- it wins on intent-style queries but loses on lexical ones. Encoder capacity alone is not the binding constraint. The synthetic intent-tag null replicates on LongMemEval (n=500) as a single-session-preference recall cliff. Adaptive vector-weight routing on LoCoMo is a measured null: 11.7pp of oracle headroom exists, but no signal we tested recovers it. All 26 result tables and 37 reproduce scripts are version-controlled and verified by a public registry; the protocol is exercised on a deterministically governed memory testbed (event-sourced decision log, DAG-state-machine schema lifecycle) so every reported CI is reproducible byte-for-byte from the ingest stream.

代理记忆检索评估嵌入模型实验协议

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。