arXiv:2609.05539cs.CVcs.AI2026-09

为视觉语言模型添加双记忆机制,提升长文本推理能力。

Dual-Latent Memory Routing for Vision-Language Reasoning

论文配图:Dual-Latent Memory Routing for Vision-Language Reasoning
图 1 · 摘自论文原文
  • 引入视觉记忆与推理记忆双通道,分别存图像证据和中间结论。
  • 动态路由机制在生成中选择性复用记忆,减少错误累积。
  • 仅增加少量参数,显著提升长序列推理表现,适合复杂视觉问答任务。

多模态大语言模型在视觉语言推理上取得进展,但生成越长,性能往往下降。主要原因是在持续增长的上下文中容易丢失早期视觉证据和中间约束。受人类解决复杂任务时区分记忆所见与所推的理解启发,我们提出DLMR——一种参数高效的机制,为多模态大模型配备双隐式记忆:一个压缩图像证据的视觉记忆,一个追踪中间结论与约束的推理记忆。通过路由器动态决定生成过程中使用哪个记忆及使用程度,既保持视觉定位,又保障长程推理连贯性。DLMR分三阶段训练,从隐式记忆构建到选择性路由器学习,同时冻结基础模型,仅引入少量可训练参数,在通用与推理基准上均取得显著提升。分析显示路由具有可解释性、状态依赖性,记忆角色专一,且长生成中解码令牌数减少。代码已开源。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) have recently made strong progress in vision-language reasoning, yet their performance often degrades as generations grow longer. A key factor is that they frequently lose track of earlier visual evidence and intermediate constraints under a monolithic growing context. Inspired by how humans separately recall what they see and what they infer when solving complex tasks, we propose DLMR, a parameter-efficient mechanism that equips MLLMs with Dual Latent Memories: a visual memory that compresses image evidence and a reasoning memory that tracks intermediate conclusions and constraints. A Router then dynamically decides which memory and how much to reuse during inference, preserving visual grounding while maintaining coherent long-horizon reasoning. DLMR is trained in three stages, from latent memory construction to selective router learning, while keeping the base MLLM frozen, yielding substantial gains on both general and reasoning benchmarks with only a small number of additional trainable parameters. Analyses further show interpretable, state-dependent routing with specialized memory roles and reduced decoding tokens over long generations. Code is available at https://github.com/Hunter-Wrynn/DLMR.

视觉语言推理增强记忆机制多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。