arXiv:2608.15517cs.CV2026-08

让视觉推理更精准:用固定查询直接定位图像细节

GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning

论文配图:GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning
图 1 · 摘自论文原文
  • 用固定查询替代逐次生成的隐状态,直接关联原始图像
  • 在5个基准上比基线模型提升5.99%~9.66%
  • 适合需要细粒度视觉理解的任务,如复杂视觉问答

链式思维已显著提升多模态大模型的问题求解能力,但细粒度视觉证据仍难以在文本推理过程中有效保留与复用。现有工具增强方法依赖外部图像重访或预定义工具,需额外计算;而连续视觉隐状态方法因自回归结构导致后续状态重复已有信息,难捕捉互补细节。本文提出GLaQ,一种基于图像原生视觉标记的固定查询框架,将中间推理转化为上下文相关的查询,并重新注入以生成答案。通过局部视图监督和任务级奖励的强化学习训练,GLaQ-7B在五个细粒度视觉理解与感知基准上相较基线模型提升5.99%至9.66%,优于所有对比的视觉隐状态方法,表明直接查询-图像对齐可无需外部操作或自回归隐状态滚动即恢复完整图像中的局部证据。

原文摘要 · Abstract (English)

Chain-of-thought reasoning has substantially improved the problem-solving capabilities of multimodal large language models. Fine-grained visual evidence, however, remains difficult to preserve and reuse across text-based reasoning steps. To address this limitation, tool-augmented thinking-with-images methods maintain visual access externally by revisiting or manipulating the image, but require predefined tools and additional inference-time processing. As an internal alternative, continuous visual latent reasoning retains intermediate computation in hidden states. However, its prevailing autoregressive construction makes each latent state depend on its predecessors, so later states may repeat information already present in the latent sequence rather than capture complementary visual details. We introduce GLaQ, a grounded latent-query framework that replaces sequential latent rollout with a fixed set of context-conditioned queries grounded in the original visual tokens. The grounded queries are reinjected for answer generation, providing direct and coordinated access to source visual evidence. We train GLaQ with localized-view supervision followed by reinforcement learning under task-level rewards. Across five benchmarks for fine-grained visual understanding and perception, GLaQ-7B gains 5.99--9.66\% over its base model and leads all compared visual latent methods, suggesting that direct query-to-image grounding can recover localized evidence from the full image without external visual operations or autoregressive latent rollouts.

视觉推理多模态大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。