测试发现大模型说要重看图时，其实没真看懂，只是在模仿说话。
Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination
- 用图像替换法检测模型是否真重看图片
- 模型换图后准确率最高降60%，基本没察觉
- 自己说要检查反而比被指令检查更不靠谱
视觉语言模型（VLMs）在推理时常说出“让我再看看图”之类的自省语句。这些语句是否引发真实的视觉重检，还是仅是习得的文本模式？我们通过视觉交换探测框架VisualSwap进行研究：在模型完成图像推理后，将其替换为视觉相似但语义不同的图像，观察模型是否能察觉。我们构建了VS-Bench数据集，包含从MathVista、MathVerse、MathVision和MMMU-Pro收集的800组图像对。在Qwen3-VL、Kimi-VL和ERNIE-VL上的实验显示惊人结果：模型普遍未能察觉图像替换，准确率下降高达60%。反直觉的是，自我反思模型比受指令模型脆弱近3倍，且规模扩大也无改善。多轮用户指令可恢复视觉一致性，但连续生成中的自动生成反思语句却无效。注意力分析表明：用户指令显著提升对视觉标记的关注，而自反思则不然。当前VLMs在声称重检视觉时，更多是‘说’而非‘看’。代码与数据集已公开于项目页：https://visualswap.github.io
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) often produce self-reflective statements like "let me check the figure again" during reasoning. Do such statements trigger genuine visual re-examination, or are they merely learned textual patterns? We investigate this via VisualSwap, an image-swap probing framework: after a model reasons over an image, we replace it with a visually similar but semantically different one and test whether the model notices. We introduce VS-Bench, 800 image pairs curated from MathVista, MathVerse, MathVision, and MMMU-Pro. Experiments on Qwen3-VL, Kimi-VL, and ERNIE-VL reveal a striking failure: models overwhelmingly miss the swap, with accuracy dropping by up to 60%. Counterintuitively, thinking models are nearly 3x more vulnerable than their instructed counterparts, and scaling offers no mitigation. Multi-turn user instructions restore visual grounding, but self-generated reflective statements during continuous generation do not. Attention analysis explains why: user instructions substantially elevate attention to visual tokens, whereas self-reflection does not. Current VLMs tend to say rather than actually see when claiming to perform visual re-examination. Our code and dataset are available at the project page: https://visualswap.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。