arXiv:2507.22925cs.CLcs.AI2025-07Conference of the …被引 56

构建分层记忆架构,让大模型更高效地长期推理。

Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents

  • 用多层抽象结构组织记忆,每层通过位置索引关联下层子记忆。
  • 在LoCoMo数据集上五项任务均超越基线,提升推理效率与连贯性。
  • 适合需要长期上下文理解的对话系统与智能代理场景。

长时记忆是影响大语言模型智能体(LLM Agents)推理能力的关键因素。将过往交互有效整合进记忆机制,可显著提升决策质量与上下文连贯性。尽管近期研究在记忆存储与检索方面取得进展,如将记忆编码为密集向量进行相似性搜索,或以图结构组织知识,但这些方法在结构化组织与高效检索方面仍存在不足。为此,我们提出一种分层记忆(H-MEM)架构,基于语义抽象程度对记忆进行多层级组织与更新。每个记忆向量嵌入指向下一层次语义相关子记忆的位置索引。推理阶段,基于索引的路由机制实现逐层高效检索,无需进行全量相似性计算。我们在LoCoMo数据集的五个任务设置上评估该方法,实验结果表明,本方法在各项任务中均持续优于五种基线方法,验证了其在长时对话场景下的有效性。

原文摘要 · Abstract (English)

Long-term memory is one of the key factors influencing the reasoning capabilities of Large Language Model Agents (LLM Agents). Incorporating a memory mechanism that effectively integrates past interactions can significantly enhance decision-making and contextual coherence of LLM Agents. While recent works have made progress in memory storage and retrieval, such as encoding memory into dense vectors for similarity-based search or organizing knowledge in the form of graph, these approaches often fall short in structured memory organization and efficient retrieval. To address these limitations, we propose a Hierarchical Memory (H-MEM) architecture for LLM Agents that organizes and updates memory in a multi-level fashion based on the degree of semantic abstraction. Each memory vector is embedded with a positional index encoding pointing to its semantically related sub-memories in the next layer. During the reasoning phase, an index-based routing mechanism enables efficient, layer-by-layer retrieval without performing exhaustive similarity computations. We evaluate our method on five task settings from the LoCoMo dataset. Experimental results show that our approach consistently outperforms five baseline methods, demonstrating its effectiveness in long-term dialogue scenarios.

记忆机制长时推理大模型智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。