arXiv:2602.16702cs.CV2026-02被引 1

提出视觉显著性感知机制,让多模态推理更稳定、少幻觉。

Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning

  • 基于视觉显著性动态选择推理路径,避免早期错误累积。
  • 在相同生成成本下,物体幻觉减少37%,推理更稳定。
  • 无需训练,适合各类视觉语言模型,尤其擅长长文本推理。

视觉语言模型(VLMs)旨在联合利用视觉与文本模态进行推理。尽管对大型语言模型(LLMs)增加推理时计算量已被证明有效,但将类似扩展应用于VLMs仍具挑战性。主要障碍在于视觉输入通常仅在生成初期提供一次,而文本推理(如早期视觉摘要)是自回归生成的,导致推理逐渐偏向文本,早期视觉定位错误会持续累积。此外,推理过程中视觉定位的原始引导往往粗略且嘈杂,难以在长文本中有效引导推理。为此,我们提出【显著性感知原则】(Saliency-Aware Principle, SAP)。SAP作用于高层次推理原则而非逐标记轨迹,可在噪声反馈下实现对离散生成的稳定控制,并在需要重新定位时允许后续推理步骤再次查阅视觉证据。此外,SAP支持多路径推理,可并行探索多样化的推理行为。SAP具有模型无关性和数据无依赖性,无需额外训练。实验证明,SAP在相当的生成代价下实现了竞争力表现,尤其显著降低物体幻觉,在推理稳定性与响应延迟方面优于传统的链式思维(CoT)长序列推理。

原文摘要 · Abstract (English)

Vision-language models (VLMs) aim to reason by jointly leveraging visual and textual modalities. While allocating additional inference-time computation has proven effective for large language models (LLMs), achieving similar scaling in VLMs remains challenging. A key obstacle is that visual inputs are typically provided only once at the start of generation, while textual reasoning (e.g., early visual summaries) is generated autoregressively, causing reasoning to become increasingly text-dominated and allowing early visual grounding errors to accumulate. Moreover, vanilla guidance for visual grounding during inference is often coarse and noisy, making it difficult to steer reasoning over long texts. To address these challenges, we propose \emph{Saliency-Aware Principle} (SAP) selection. SAP operates on high-level reasoning principles rather than token-level trajectories, which enable stable control over discrete generation under noisy feedback while allowing later reasoning steps to re-consult visual evidence when renewed grounding is required. In addition, SAP supports multi-route inference, enabling parallel exploration of diverse reasoning behaviors. SAP is model-agnostic and data-free, requiring no additional training. Empirical results show that SAP achieves competitive performance, especially in reducing object hallucination, under comparable token-generation budgets while yielding more stable reasoning and lower response latency than CoT-style long sequential reasoning.

多模态推理视觉语言模型幻觉抑制推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。