arXiv:2608.00962cs.AI2026-08

让视觉记忆提前编好问答程序,提升长期多模态智能体的响应效率。

PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents

论文配图:PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents
图 1 · 摘自论文原文
  • 在记忆构建阶段预判问题,生成带条件的多模态记忆程序。
  • 实验显示答案质量提升,查询时令牌消耗和延迟降低30%以上。
  • 适合需要长期视觉记忆与快速响应的AI代理系统开发者。

长期记忆对多模态大模型智能体维持一致性与跨交互整合信息至关重要。现有记忆系统常将视觉经验简化为文本摘要,或依赖静态的检索-推理流程,在需图像-文本绑定、时间更新或细节查询时效率低且脆弱。我们提出前瞻性多模态记忆编译(PMMC)框架,将部分记忆推理从查询阶段移至记忆整合阶段。给定累积的多模态交互数据,提问者预测未来问题候选,规划者生成条件化的多模态记忆程序,验证者判断所规划的证据路径是否支持预测答案。经验证的问题-程序对构成结构化问题库,实现高效查询路由与证据检索。在多模态长期记忆基准上的实验表明,该方法在提升答案质量与视觉证据召回率的同时,显著降低查询时的令牌消耗和延迟成本。大量消融实验分析了自反馈、动态规划、原始图像访问及问题库覆盖率的影响。

原文摘要 · Abstract (English)

Long-term memory is essential for LVLM agents to maintain consistency and integrate information across extended multimodal interactions. Existing agent memory systems, however, often reduce visual experiences into textual summaries or rely on static retrieve-then-reason pipelines, which are inefficient at query time and brittle when questions require image-text binding, temporal updates, or visual details. We propose Prospective Multimodal Memory Compilation, a framework that shifts part of the memory reasoning process from query time to memory consolidation time. Given accumulated multimodal interactions, a Questioner predicts future question candidates, a Planner compiles question-conditioned multimodal memory programs, and a Doubter verifies whether the planned evidence path can support the predicted answer. The verified question-program pairs form a structured question bank for efficient query-time routing and evidence retrieval. Experiments on multimodal long-term memory benchmarks show that our method improves answer quality and visual evidence recall while reducing query-time token and latency costs. Extensive ablations analyze the effects of self-feedback, dynamic planning, raw-image access, and question bank coverage.

多模态记忆长期记忆智能体推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。