arXiv:2602.21497cs.CV2026-02中稿 · ed被引 2

不训练模型,通过视觉证据迭代校验推理步骤,减少幻觉。

See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs

  • 测试时用视觉证据动态校验每步推理,确保与图像一致
  • 在TreeBench上提升16.5%-29.5%,RH-Bench的RH-AUC提高13.7%
  • 无需训练、可适配多种模型,适合部署在现有LVLM上

近期大型视觉语言模型(LVLMs)通过生成长链式思维(CoT)展现出强大推理能力。然而,多模态场景下的CoT推理极易受视觉幻觉传播影响:一旦中间推理步骤与视觉证据不符,后续即使逻辑正确,仍会导致错误最终答案。现有方法通过强化学习让模型“结合图像思考”,但成本高、依赖特定模型且泛化性差。本文提出一种轻量级、无需训练、即插即用的迭代框架,实现视觉锚定的多模态推理。核心思想是在测试时以视觉证据逐步监督推理过程,确保每个生成的词元均有对应视觉线索支持。具体地,构建文本-视觉证据池引导推理生成;当现有证据不足时,视觉决策模块基于当前推理上下文从图像中动态提取新相关证据,扩展证据池直至模型获得足够视觉确定性,从而终止推理并输出最终答案。在多个LVLM骨干模型和基准上的实验表明,该方法在TreeBench上取得16.5%-29.5%的性能提升,在RH-Bench上实现13.7%的RH-AUC增益,显著降低幻觉率的同时提升推理准确性,且无需额外训练。

原文摘要 · Abstract (English)

Recent large vision-language models (LVLMs) have demonstrated impressive reasoning ability by generating long chain-of-thought (CoT) responses. However, CoT reasoning in multimodal contexts is highly vulnerable to visual hallucination propagation: once an intermediate reasoning step becomes inconsistent with the visual evidence, subsequent steps-even if logically valid-can still lead to incorrect final answers. Existing solutions attempt to mitigate this issue by training models to "think with images" via reinforcement learning (RL). While effective, these methods are costly, model-specific, and difficult to generalize across architectures. Differently, we present a lightweight method that bypasses RL training and provides an iterative, training-free, plug-and-play framework for visually-grounded multimodal reasoning. Our key idea is to supervise each reasoning step at test time with visual evidence, ensuring that every decoded token is justified by corresponding visual cues. Concretely, we construct a textual visual-evidence pool that guides the model's reasoning generation. When existing evidence is insufficient, a visual decider module dynamically extracts additional relevant evidence from the image based on the ongoing reasoning context, expanding the pool until the model achieves sufficient visual certainty to terminate reasoning and produce the final answer. Extensive experiments on multiple LVLM backbones and benchmarks demonstrate the effectiveness of our approach. Our method achieves 16.5%-29.5% improvements on TreeBench and 13.7% RH-AUC gains on RH-Bench, substantially reducing hallucination rates while improving reasoning accuracy without additional training.

多模态推理视觉幻觉链式思维零训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。