让多模态大模型在推理时持续关注视觉信息，提升复杂问题解决能力。
Vision-aligned Latent Reasoning for Multi-modal Large Language Model
- 每步推理前生成与视觉对齐的隐向量，引导模型基于感知线索思考。
- 在VSI-Bench上性能从33.0%提升至52.9%，超越Qwen2.5-VL 19.9个百分点。
- 适合需要长链推理与精确视觉理解的任务，如复杂图文问答。
尽管多模态大语言模型（MLLMs）在各类理解任务上取得进展，但在需要多步推理的问题上仍表现不佳，主要因长上下文生成过程中视觉信息逐渐丢失，限制了其测试时扩展能力。为此，我们提出视觉对齐隐式推理（VaLR），一种简单而有效的推理框架：在每次思维链推理前动态生成与视觉对齐的隐向量，引导模型基于潜在空间中的感知线索进行推理。具体而言，通过将MLLM中间嵌入与视觉编码器嵌入对齐，训练模型在推理过程中保留视觉知识。实证结果表明，VaLR在多个需长上下文理解或精确视觉感知的基准上持续优于现有方法，并展现出先前MLLM中未见的测试时扩展行为。尤其在VSI-Bench上，性能从33.0%显著提升至52.9%，较Qwen2.5-VL提升19.9个百分点。
原文摘要 · Abstract (English)
Despite recent advancements in Multi-modal Large Language Models (MLLMs) on diverse understanding tasks, these models struggle to solve problems which require extensive multi-step reasoning. This is primarily due to the progressive dilution of visual information during long-context generation, which hinders their ability to fully exploit test-time scaling. To address this issue, we introduce Vision-aligned Latent Reasoning (VaLR), a simple, yet effective reasoning framework that dynamically generates vision-aligned latent tokens before each Chain of Thought reasoning step, guiding the model to reason based on perceptual cues in the latent space. Specifically, VaLR is trained to preserve visual knowledge during reasoning by aligning intermediate embeddings of MLLM with those from vision encoders. Empirical results demonstrate that VaLR consistently outperforms existing approaches across a wide range of benchmarks requiring long-context understanding or precise visual perception, while exhibiting test-time scaling behavior not observed in prior MLLMs. In particular, VaLR improves the performance significantly from 33.0% to 52.9% on VSI-Bench, achieving a 19.9%p gain over Qwen2.5-VL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。