提出可自我反思的检索记忆框架,提升长视频多模态推理的准确性。
RRM: Experience-Driven Reflective Retrieval Memory for Long-Horizon Multimodal Reasoning

- 用历史任务经验提炼通用检索策略,指导当前搜索
- 在三个长视频基准上超越现有最优方法
- 适合需要长期记忆与复杂推理的多模态系统
现有多模态长期记忆代理依赖外部记忆来应对长视频的上下文限制,但多数方法关注存储内容而非如何检索。当检索失败时,现有代理缺乏诊断过往轨迹并调整未来策略的能力。本文提出反射式检索记忆(RRM),在以实体为中心的多模态记忆图基础上,引入从历史任务轨迹中提炼的可迁移检索经验记忆。不同于保存当前视频事实证据的情景记忆和语义记忆,反射经验记忆捕捉跨任务复用的搜索策略。RRM将检索到的经验转化为查询级指导,而答案生成仍仅依赖从当前视频新检索到的事实证据。生命周期管理机制通过使用频率、重用反馈和时间衰减调控经验记忆,减少冗余与噪声。RRM在M3-Bench-Robot、M3-Bench-Web和Video-MME-Long三个基准上持续优于先前最先进方法,验证了反射式检索记忆在长时程多模态推理中的有效性。
原文摘要 · Abstract (English)
Existing multimodal long-term memory agents use external memory to overcome the limited context available for long videos. However, most methods emphasize what to store rather than how stored memory should be retrieved. When retrieval becomes inaccurate or repeatedly fails to obtain useful evidence, existing agents lack mechanisms to diagnose failures from previous task trajectories and adapt future search strategies.We introduce Reflective Retrieval Memory (RRM), a reflective memory framework for long-horizon multimodal reasoning. RRM augments an entity-centric multimodal memory graph with reflective experience memory, which distills transferable procedural retrieval knowledge from historical task trajectories. Unlike episodic and semantic memories that preserve factual evidence from the current video, reflective experience memory captures reusable search strategies across tasks. RRM converts retrieved experiences into query-level guidance, while answer generation remains conditioned only on factual evidence newly retrieved from the current video. A lifecycle management mechanism further regulates experience memory through usage frequency, reuse feedback, and temporal decay, thereby reducing redundancy and noise. RRM consistently outperforms previous state-of-the-art approaches on M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long, demonstrating the effectiveness of reflective retrieval memory for long-horizon multimodal reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。