对比四种记忆系统在多轮任务中的表现，发现结构化记忆在数学物理任务中更优。
MemoryLake on MemoryArena: A Matched Study of Agent Memory Backends
- 采用统一框架测试四种记忆后端，比较其对多轮任务的支持能力
- 结构化记忆在数学、物理和渐进检索任务中成功率最高，达9/40、12/20
- 结果表明记忆设计需匹配任务类型，适合研究智能体记忆系统的研究者
多数智能体记忆评测仅测试事后召回，而MemoryArena评估记忆是否支持跨会话、相互依赖的任务完成。我们对比了结构化多轨记忆系统MemoryLake、基于text-embedding-3-small向量的RAG系统Mem0、长上下文控制方案，在所有五个MemoryArena领域中的表现。四套系统使用相同的代理框架、gpt-5-mini模型别名、任务样本与评分代码，唯一变量是记忆集成模块。由于每个后端均整合了写入、检索、融合、预算和提示组装策略，本研究为系统级匹配对比，非仅表示层消融或成本匹配实验。在共享评估集上，MemoryLake在数学（9/40）、物理（12/20）和渐进检索（4/20）任务中成功率最高；旅行规划全为0成功率，网络购物仅长上下文实现1/150成功，MemoryLake在旅行软流程得分和购物步骤匹配上位列第三。按MemoryArena的全局平均规则，五项成功率均值为20.5%（MemoryLake） vs 13.6%（最优对照组）。这些为点估计：样本量有限，置信区间重叠，未报告配对显著性检验。单独运行MemoryLake处理全部221个渐进查询，失败计数成功率26.7%（59/221），但不构成基准对比。结果支持记忆后端依工作负载而异的观点，并在共享数据集上表现出领先，但不确立基准整体最优或结构表示的因果优势。
原文摘要 · Abstract (English)
Most agent-memory benchmarks test post-hoc recall, whereas MemoryArena evaluates whether memory supports interdependent, multi-session task completion. We compare MemoryLake, a structured multi-track memory backend, with Mem0, text-embedding-3-small vector RAG, and a long-context control across all five MemoryArena domains. The systems share the same agent framework, requested gpt-5-mini model alias, task samples, and scoring code; the memory integration is the intentionally changed component. Because each backend bundles write, retrieval, consolidation, budgeting, and prompt-assembly choices, the study is a matched system-level comparison, not a representation-only ablation or a cost-matched experiment. On the shared evaluation sets, MemoryLake has the highest observed success rate (SR) in mathematics (9/40), physics (12/20), and progressive retrieval (4/20). Every system has zero SR in travel planning, and web shopping yields a single bundle-level success (long context, 1/150); MemoryLake ranks third on both the travel soft process score and shopping step match. Following MemoryArena's suite-level convention, a post-hoc equal-weight average over the five SRs is 20.5% for MemoryLake versus 13.6% for the best comparator. These are point estimates: sample sizes are modest, confidence intervals overlap, and we do not report paired significance tests. A separate MemoryLake-only run over all 221 progressive queries yields a failure-counted SR of 26.7% (59/221) and is not a baseline comparison. The results support a workload-dependent view of memory backends and an observed lead among the four evaluated systems on the shared sets; they do not establish benchmark-wide state of the art or a causal advantage of representation structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。