arXiv:2606.24775cs.CLcs.DB2026-06被引 8

从数据管理视角系统评估大模型智能体记忆系统,揭示其性能与工作负载的匹配关系。

Are We Ready For An Agent-Native Memory System?

论文配图:Are We Ready For An Agent-Native Memory System?
图 1 · 摘自论文原文
  • 将记忆系统拆分为表示存储、提取、检索路由和维护四模块进行分析
  • 12种系统在5个任务上表现各异,无统一最优架构
  • 局部维护比全局重构更节省成本,适合实际部署

大语言模型智能体的记忆能力已从简单的检索增强机制演变为支持持久化存储、检索、更新、整合及动态生命周期管理的数据管理系统。然而,现有评估仍主要依赖端到端任务成功率(如F1、BLEU),将底层系统视为黑箱,导致操作成本、模块间架构权衡以及动态知识更新下的鲁棒性等关键问题未被充分探讨。本文从数据管理角度开展系统性实验研究,提出一个将记忆系统分解为四个核心模块的分析框架:记忆表示与存储、提取、检索与路由、维护。基于该框架,我们在涵盖11个数据集的五个基准工作负载上评估了12种代表性记忆系统及两种基线。大规模端到端测试表明,没有单一架构在所有场景下占优,效果高度依赖于记忆结构与工作负载瓶颈的匹配程度。通过细粒度消融实验,量化了各模块对表示保真度、检索精度、更新正确性和长时稳定性的影响。最后,在真实工作负载下揭示了成本-性能权衡,发现局部维护比全局重组更具成本效益。基于这些发现,我们指出了构建真正智能体原生记忆系统的潜在方向。代码已公开于 https://github.com/OpenDataBox/MemoryData。

原文摘要 · Abstract (English)

Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.

智能体记忆数据管理系统评估架构优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。