让视觉语言模型回答时真正依赖图像证据，而非靠猜。
Evidence-RL: Towards Evidence-intensive Visual Reasoning
- 用反事实方法检测答案是否真由局部图像证据支持
- 在9个数据集上超越已有强化学习方法，提升模型可信度
- 无需标注证据，训练后推理无额外开销，适合通用模型优化
视觉语言模型应基于具体图像证据作答，而非依赖语言先验、数据集捷径或无关视觉上下文。现有感知敏感的后训练方法通过全局扰动或注意力代理促进图像使用，但无法验证答案是否因果性地依赖支持它的局部证据。本文提出反事实证据解耦（CED），一种训练期的模型接地审计机制。针对每个回答，CED中和以物体为中心的证据区域，并比较其支持度下降与对应非证据区域的差异。该信号结合答案正确性融入GRPO，奖励那些依赖证据路径而非捷径或干扰路径的正确回答。CED采用弱监督物体级提议，无需问题特定的证据标注，且不增加推理开销。在九个公开基准和四个骨干网络上，CED均优于先前基于强化学习的后训练方法，目标分析验证了其物体中心信号的有效性。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context. Existing perception-aware post-training methods encourage image use through global perturbations or attention proxies, but they do not test whether a sampled answer causally depends on the local evidence that supports it. We propose Counterfactual Evidence Disentanglement (CED), a training-time evidence audit for VLM grounding. For each response, CED neutralizes an object-centric Evidence Region and compares the resulting support drop against matched non-evidence Regions. We combine this signal with answer correctness inside GRPO, rewarding correct answers that rely on the evidence path rather than shortcut or nuisance paths. CED uses weak object-level proposals, requires no question-specific evidence annotations, and adds no inference-time overhead. Across nine public benchmarks and four backbones, CED outperforms prior RL-based post-training methods, with targeted analyses verifying its object-centric signal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。