arXiv:2605.18641cs.CV2026-05被引 1

让视觉推理在无声中生效:用注意力奖励提升隐式视觉引导能力

Leveraging Latent Visual Reasoning in Silence

论文配图:Leveraging Latent Visual Reasoning in Silence
图 1 · 摘自论文原文
  • 用注意力奖励机制引导生成的隐式视觉标记与后续文本交互
  • 即使推理时极少生成隐式标记,仍显著提升感知与视觉推理性能
  • 揭示隐式视觉推理的价值在于训练期引导而非推理时存在

隐式视觉推理通过在文本生成前插入连续隐式标记,更直接地利用视觉证据进行多模态推理。然而这些隐式标记在推理阶段是否必要尚不明确。我们发现,用随机噪声替代或完全移除隐式标记,在空间推理基准上几乎不造成性能下降。强化学习进一步削弱了后训练阶段的隐式生成行为。这引发核心问题:隐式视觉推理是否仍有意义?我们认为其价值应体现在对学习过程的引导效果,而非推理时是否显式存在。分析表明,隐式推理在不同题型上表现不均,但任务级路由策略脆弱。为此,我们提出一种基于注意力的奖励机制,鼓励生成的隐式标记在强化学习中与后续文本标记互动。该奖励使隐式模式激活时更具利用性,同时保留纯文本推理的灵活性。实验显示,该方法在感知和视觉推理基准上均实现性能提升,即便后训练阶段隐式标记极少生成。结果表明,即使推理时不显式表达,隐式视觉推理也能在沉默中塑造更强的视觉定位与更准确的文本推理。代码与模型已公开于 GitHub 与 Hugging Face。

原文摘要 · Abstract (English)

Latent visual reasoning involves visual evidence more directly in multimodal reasoning by inserting continuous latent tokens before textual generation. However, the necessity of these latent tokens at inference remains ambiguous. We show that replacing latent tokens with random noise or removing them completely causes little performance degradation across spatial reasoning benchmarks. Reinforcement learning further diminishes the latent generation behavior after post-training. These observations raise a central question: Is latent visual reasoning still meaningful? We argue that its value should be measured by how effectively latent tokens guide learning, rather than whether they persist as an inference-time format. Our analysis shows that latent reasoning is unevenly favorable across question types, yet hard task-level routing for applying latent generation is brittle. Motivated by these findings, we propose an attention-based reward that encourages generated latent tokens to interact with later text tokens during RL. This reward promotes latent utilization when the latent mode is activated while preserving the flexibility to use pure-text reasoning. Experiments show that our method improves performance across perception and visual reasoning benchmarks, even when latent tokens are rarely generated after post-training. Our results highlight that, without explicit expression at inference, latent visual reasoning can shape better visual grounding and more accurate textual reasoning in silence. Our code and trained models are publicly available at \href{https://github.com/ddydyd32/silent-lvr/tree/master}{GitHub} and \href{https://huggingface.co/collections/cornuHGF/silent-lvr}{Hugging Face}.

视觉推理隐式表示强化学习多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。