构建可长期维持的多模态记忆系统,让AI更懂用户长期互动。
TeleMem: Building Long-Term and Multimodal Memory for Agentic AI
- 通过叙事动态提取保留对话相关记忆,避免无关信息堆积。
- 批量处理与聚类压缩记忆,减少43%令牌使用,提速2.1倍。
- 支持视频等多模态理解,适合长时角色扮演类AI应用。
大型语言模型在许多自然语言任务中表现优异,但在长时间对话中因注意力受限而难以持续交互。检索增强生成(RAG)虽缓解此问题,但缺乏可靠的记忆更新与精炼机制,导致模式驱动的幻觉、写入效率低,且不支持多模态推理。为此,我们提出TeleMem,一个统一的长期多模态记忆系统,通过叙事动态提取保持用户档案连贯性,仅保留基于对话的信息。TeleMem引入结构化写入流程,对记忆条目进行批处理、检索、聚类与合并,显著提升存储效率,减少令牌消耗,并加速内存操作。此外,结合多模态记忆模块与ReAct式推理,系统具备闭环观察-思考-行动能力,可在长期上下文中准确理解复杂视频内容。实验表明,在ZH-4O长期角色扮演游戏基准上,TeleMem相较最先进基线Mem0,准确率提升19%,令牌用量减少43%,速度提升2.1倍。
原文摘要 · Abstract (English)
Large language models (LLMs) excel at many NLP tasks but struggle to sustain long-term interactions due to limited attention over extended dialogue histories. Retrieval-augmented generation (RAG) mitigates this issue but lacks reliable mechanisms for updating or refining stored memories, leading to schema-driven hallucinations, inefficient write operations, and minimal support for multimodal reasoning.To address these challenges, we propose TeleMem, a unified long-term and multimodal memory system that maintains coherent user profiles through narrative dynamic extraction, ensuring that only dialogue-grounded information is preserved. TeleMem further introduces a structured writing pipeline that batches, retrieves, clusters, and consolidates memory entries, substantially improving storage efficiency, reducing token usage, and accelerating memory operations. Additionally, a multimodal memory module combined with ReAct-style reasoning equips the system with a closed-loop observe, think, and act process that enables accurate understanding of complex video content in long-term contexts. Experimental results show that TeleMem surpasses the state-of-the-art Mem0 baseline with 19% higher accuracy, 43% fewer tokens, and a 2.1x speedup on the ZH-4O long-term role-play gaming benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。