用树状结构动态管理长时记忆，提升大模型推理能力
From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs
- 构建动态树形记忆结构，分层存储语义信息
- 在多轮对话与文档问答任务中显著提升表现
- 适合需要长期记忆与复杂推理的场景
大语言模型上下文窗口虽已大幅扩展，但长期记忆管理仍存挑战。我们提出MemTree算法，采用动态树形记忆结构优化信息的组织、检索与整合，类似人类认知中的知识框架。该结构分层存储文本内容、语义嵌入及不同抽象层级的信息。通过计算并比较新旧信息的语义嵌入，动态调整记忆结构，增强模型上下文感知能力。相比传统扁平查找表的记忆增强方法，MemTree在多轮对话理解与文档问答基准测试中表现更优，显著提升了复杂推理与长时交互的处理效率。
原文摘要 · Abstract (English)
Recent advancements in large language models have significantly improved their context windows, yet challenges in effective long-term memory management remain. We introduce MemTree, an algorithm that leverages a dynamic, tree-structured memory representation to optimize the organization, retrieval, and integration of information, akin to human cognitive schemas. MemTree organizes memory hierarchically, with each node encapsulating aggregated textual content, corresponding semantic embeddings, and varying abstraction levels across the tree's depths. Our algorithm dynamically adapts this memory structure by computing and comparing semantic embeddings of new and existing information to enrich the model's context-awareness. This approach allows MemTree to handle complex reasoning and extended interactions more effectively than traditional memory augmentation methods, which often rely on flat lookup tables. Evaluations on benchmarks for multi-turn dialogue understanding and document question answering show that MemTree significantly enhances performance in scenarios that demand structured memory management.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。