arXiv:2505.19549cs.CL2025-05被引 21

让对话模型学会多粒度记忆关联,提升长期对话连贯性。

From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents

  • 构建多粒度记忆单元,用高斯混合模型关联新旧记忆。
  • 实验显示在4个基准上优于现有方法,不同查询类型均表现更优。
  • 适合需要长时记忆的智能客服、虚拟助手等场景。

大型语言模型在对话系统中广泛应用,但用户与代理之间日益增长的交互积累了大量对话记录,而有限的上下文窗口使模型难以维持连贯的长期记忆并提供个性化回应。尽管检索增强型记忆系统已出现,但现有方法通常依赖单一粒度的记忆分割与检索,难以捕捉深层记忆关联,导致有用信息遗漏或噪声过多,影响性能。为此,我们提出MemGAS框架,通过构建多粒度关联、自适应选择与检索机制,增强记忆整合能力。该框架基于多粒度记忆单元,利用高斯混合模型对新记忆与历史记忆进行聚类与关联。基于熵的路由机制通过评估查询相关性分布,自适应选择最优粒度,平衡信息完整性和噪声。检索结果进一步通过大模型过滤优化。在四个长期记忆基准上的实验表明,MemGAS在问答与检索任务上均优于当前最先进方法,跨不同查询类型和top-K设置表现更佳。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have recently been widely adopted in conversational agents. However, the increasingly long interactions between users and agents accumulate extensive dialogue records, making it difficult for LLMs with limited context windows to maintain a coherent long-term dialogue memory and deliver personalized responses. While retrieval-augmented memory systems have emerged to address this issue, existing methods often depend on single-granularity memory segmentation and retrieval. This approach falls short in capturing deep memory connections, leading to partial retrieval of useful information or substantial noise, resulting in suboptimal performance. To tackle these limits, we propose MemGAS, a framework that enhances memory consolidation by constructing multi-granularity association, adaptive selection, and retrieval. MemGAS is based on multi-granularity memory units and employs Gaussian Mixture Models to cluster and associate new memories with historical ones. An entropy-based router adaptively selects optimal granularity by evaluating query relevance distributions and balancing information completeness and noise. Retrieved memories are further refined via LLM-based filtering. Experiments on four long-term memory benchmarks demonstrate that MemGAS outperforms state-of-the-art methods on both question answer and retrieval tasks, achieving superior performance across different query types and top-K settings. \footnote{https://github.com/quqxui/MemGAS}

对话系统长时记忆多粒度记忆增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。