arXiv:2608.04385cs.CV2026-08中稿 · ACM Multimedia 202…

让视觉语言模型在多步推理中找回图像依据,避免越推越偏。

ReGround: Restoring Visual Grounding in Multi-Step Reasoning through Self-Diagnosis and Visual Re-Examination

论文配图:ReGround: Restoring Visual Grounding in Multi-Step Reasoning through Self-Diagnosis and Visual Re-Examination
图 1 · 摘自论文原文
  • 通过自诊断和图像重检两阶段机制,识别并修复推理中的视觉失准
  • 在8个基准上实现显著提升,关键任务最高增益达数个百分点
  • 无需修改模型结构,适合需精准视觉推理的应用场景

视觉语言模型在多步推理中常丢失视觉依据:随着推理链增长,后续步骤越来越依赖语言先验而非图像证据。我们在四个基准共2510个样本上发现一致现象——首轮推理中图像令牌注意力熵下降,图像重注入后再次上升。但有效重检需两个要素:图像重注入与针对性自诊断。缺乏诊断时重检可能损害性能;准确诊断则带来显著提升,在关键基准上提升数个百分点,表明诊断质量是决定重检成败的关键。我们提出ReGround,一种无需架构修改或外部工具的两阶段框架,使模型能自主诊断视觉失准并选择性重检图像证据。通过能力蒸馏,更强版本仅在数据构建时提供诊断支持,策略模型在推理时独立完成诊断并保留大部分增益。在两种主流视觉语言模型上的八个基准测试中,均取得持续改进,尤其在视觉密集型多步推理任务表现突出,推理开销远低于工具增强基线。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) often lose visual grounding during multi-step reasoning: as reasoning chains grow longer, later inference steps rely increasingly on language priors rather than image evidence. We identify a consistent benchmark-level signature associated with this degradation: across 2,510 re-examined samples from four benchmarks, attention entropy over image tokens typically decreases during Round 1 and rises again after image re-injection. However, we find that effective visual re-examination requires two complementary ingredients: image re-injection and targeted self-diagnosis. Without targeted diagnosis, re-examination can even hurt performance, whereas accurate self-diagnosis yields substantial gains -- a swing of several points on key benchmarks, indicating that diagnostic quality is a key factor in whether re-examination helps or hurts in our setting. We present ReGround, a two-stage framework that teaches VLMs to self-diagnose grounding failures and selectively re-examine visual evidence, without architectural modifications or external tools. Through capability bootstrapping, a stronger variant from the same model family provides diagnostic scaffolding only during data construction, while the policy model learns to diagnose autonomously at inference time and retains most of the assisted gains. Experiments on eight benchmarks across two VLM backbones demonstrate consistent gains, especially on visually intensive multi-step reasoning tasks, while incurring only modest inference overhead relative to tool-augmented baselines. Project page: https://sespoir.github.io/reground-page/ . Code: https://github.com/sespoir/ReGround .

视觉推理多步推理自诊断图像重检

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。