arXiv:2602.07624cs.AI2026-02被引 13

让对话模型持续学习用户习惯,支持长期个性化交互。

M2A: Multimodal Memory Agent with Dual-Layer Hybrid Memory for Long-Term Personalized Interactions

  • 双层记忆系统:保留原始对话与高层语义信息,支持动态更新。
  • 在线学习机制使模型在数周对话中持续吸收用户新偏好,性能显著提升。
  • 适合长期多模态交互场景,如智能助手、个性化客服。

本文针对长期人机交互中的个性化问答难题:当对话历史跨越数周或数月并超出上下文窗口时,现有个性化机制难以持续吸收用户的增量概念、别名和偏好。当前个性化多模态模型多为静态,概念在初始化后固定不变。为此,我们提出M2A——一种基于双层混合记忆的代理系统,通过在线更新维护个性化多模态信息。系统包含两个协作代理:ChatAgent负责管理用户交互并自主决定何时查询或更新记忆;MemoryManager将请求分解为对双层记忆库的具体操作,该库结合了不可变的RawMessageStore(原始对话日志)与高层语义的SemanticMemoryStore(语义记忆库),提供不同粒度的记忆。此外,我们构建了一个可复用的数据合成管道,将Yo'LLaVA和MC-LLaVA中的概念引导会话注入LoCoMo长对话中,同时保持时间一致性。实验表明,M2A显著优于基线方法,证明将个性化从一次性配置转变为协同演化的记忆机制,是实现高质量长期多模态个性化响应的可行路径。代码已开源:https://github.com/Little-Fridge/M2A。

原文摘要 · Abstract (English)

This work addresses the challenge of personalized question answering in long-term human-machine interactions: when conversational history spans weeks or months and exceeds the context window, existing personalization mechanisms struggle to continuously absorb and leverage users' incremental concepts, aliases, and preferences. Current personalized multimodal models are predominantly static-concepts are fixed at initialization and cannot evolve during interactions. We propose M2A, an agentic dual-layer hybrid memory system that maintains personalized multimodal information through online updates. The system employs two collaborative agents: ChatAgent manages user interactions and autonomously decides when to query or update memory, while MemoryManager breaks down memory requests from ChatAgent into detailed operations on the dual-layer memory bank, which couples a RawMessageStore (immutable conversation log) with a SemanticMemoryStore (high-level observations), providing memories at different granularities. In addition, we develop a reusable data synthesis pipeline that injects concept-grounded sessions from Yo'LLaVA and MC-LLaVA into LoCoMo long conversations while preserving temporal coherence. Experiments show that M2A significantly outperforms baselines, demonstrating that transforming personalization from one-shot configuration to a co-evolving memory mechanism provides a viable path for high-quality individualized responses in long-term multimodal interactions. The code is available at https://github.com/Little-Fridge/M2A.

多模态长期记忆个性化对话系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。