对比多种记忆存储方式,发现不同场景下各有优劣。
Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents

- 系统测试7类记忆介质在不同任务中的表现
- 长上下文问答靠检索,但决策任务过度检索会失效
- 适合动态切换记忆策略的智能系统设计
记忆正成为长周期大模型智能体的核心基础设施,但现有评估难以指导在不同运行环境下选择何种记忆介质(即记忆的存储与表示载体)。我们构建了一个统一的基准测试框架,对增强记忆的大模型智能体所使用的七类记忆介质进行系统性评估:密集/稀疏索引、文本记录、结构化存储、分层存储、基于优化的记忆、参数化更新以及激活兼容的上下文机制。实验覆盖三类主干模型和四套评测集,涵盖用户导向的问答与智能体导向的决策任务,共测量26项性能与效率指标。结果表明:无单一介质始终最优;广泛检索有利于长上下文事实问答,但过度检索会因分散注意力而损害序列决策能力;可扩展性引入新维度,某些在中等历史长度表现良好的介质在超长序列下可能成本剧增或变得脆弱。这些发现支持将介质路由作为自适应记忆系统的必要组件,并为设计高效、可靠、环境感知的长期记忆提供实证依据。代码将在论文接收后公开。
原文摘要 · Abstract (English)
Memory is becoming core infrastructure for long-horizon LLM agents, yet existing evaluations offer limited guidance on which memory substrate, namely the underlying medium in which memory is represented and stored, should be used under different operating regimes. We present a controlled harness evaluation of memory substrates for memory-augmented agents, covering dense and sparse indices, text records, structural stores, hierarchical stores, refinement-based memories, parametric updates, and activation-compatible context mechanisms. Across three backbone models and four benchmark suites spanning user-centric question answering and agent-centric decision-making, we instrument 26 performance and efficiency metrics under a unified harness. Our results show that no single substrate consistently dominates: broad retrieval benefits long-context factual QA, while excessive retrieval can harm sequential decision-making by shifting attention away from action-critical context. Scalability introduces a further routing axis, as substrates that perform well at moderate history lengths can become costly or brittle at longer horizons. These findings motivate substrate routing as a necessary component of adaptive agent memory systems and provide empirical guidance for designing efficient, reliable, and regime-aware long-term memory for LLM agents. Code will be made available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。