让模型必须通过隐状态推理图像,避免依赖显式文本链。
Reason Through the Latent! Making Latent Visual Reasoning Necessary

- 用递归更新隐状态,强制模型依赖图像隐含信息
- 在多个基准上保持强性能,而其他方法失效
- 适合研究视觉推理机制或模型可解释性
隐状态视觉推理旨在通过隐藏状态计算实现多模态推理,而非依赖显式的文本思维链。然而,隐状态中存在视觉信息,并不意味着模型实际使用该状态进行预测,尤其当仍存在其他图像条件路径时。本文提出因果视觉递归推理(CVRR),在保留预训练视觉能力的同时,使递归计算成为预测所必需的图像条件路径。CVRR 在预训练视觉语言模型融合图像后,从问题隐藏状态初始化递归,反复更新该状态并重读同一固定视觉证据。解码前移除视觉状态和原始多模态键值缓存,确保仅最终递归状态携带图像条件信息。在 V^*、MMVP、BLINK 与 MME-RealWorld-Lite 基准上,CVRR 在此严格接口下仍保持优异表现,而兼容的隐状态推理器即使重训也无法恢复相当视觉能力。因果干预进一步表明,当问题固定时,预测仍对递归内容敏感,且持续视觉证据会因果地修正递归轨迹。这些结果区分了隐状态的信息丰富性与实际用于预测的计算过程。
原文摘要 · Abstract (English)
Latent visual reasoning aims to perform multimodal reasoning through hidden-state computation rather than explicit textual chains of thought. However, visual information being present in a latent state does not imply that the model actually relies on that state when producing its answer, especially when alternative image-conditioned paths remain available. We introduce \textbf{C}ausal \textbf{V}isual \textbf{R}ecurrent \textbf{R}easoning (CVRR), which preserves pretrained visual competence while making recurrent computation the required image-conditioned path to prediction. CVRR initializes recurrence from the question hidden state after the pretrained vision-language model has incorporated the image, then repeatedly updates this state while re-reading the same fixed visual evidence. Before decoding, visual states and the original multimodal KV cache are removed so that only the final recurrent state carries image-conditioned information to the answer. Across the $V^*$, MMVP, BLINK, and MME-RealWorld-Lite benchmarks, CVRR retains strong performance under this strict interface, while compatible latent reasoners fail to recover comparable visual competence even when retrained under the same constraint. Causal interventions further show that predictions remain sensitive to recurrent content when the question is held fixed, and that persistent visual evidence causally revises the recurrent trajectory. These results distinguish latent informativeness from latent computation that is actually used for prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。