发现视觉语言模型会编造图像描述,可能误导医疗诊断。
MIRAGE: The Illusion of Visual Understanding
- 模型在无图情况下仍能生成详细推理,称作'幻觉推理'
- 未见图像时在胸部X光题答榜上获得第一名
- 明确要求猜测时表现下降,说明其依赖虚假图像假设
多模态AI系统在真实任务中表现优异,但其视觉-语言推理机制仍不清晰。我们发现三个挑战主流认知的现象:第一,前沿模型能对从未见过的图像生成详尽描述和病理推断,称为‘幻觉推理’;第二,即使没有图像输入,模型在通用与医学多模态基准上仍取得极高分数,极端案例中在标准胸片问答任务中排名第一;第三,当明确指令要求猜测而非默认有图像时,性能显著下降。这表明模型倾向于‘假设有图像’,形成保守响应模式。这些发现揭示了视觉-语言模型在推理与评估中的根本缺陷,亟需去除文本线索的私有基准,尤其在医疗领域。我们提出B-Clean作为公正、以视觉为依据的多模态评估方案。
原文摘要 · Abstract (English)
Multimodal AI systems have achieved remarkable performance across a broad range of real-world tasks, yet the mechanisms underlying visual-language reasoning remain surprisingly poorly understood. We report three findings that challenge prevailing assumptions about how these systems process and integrate visual information. First, Frontier models readily generate detailed image descriptions and elaborate reasoning traces, including pathology-biased clinical findings, for images never provided; we term this phenomenon mirage reasoning. Second, without any image input, models also attain strikingly high scores across general and medical multimodal benchmarks, bringing into question their utility and design. In the most extreme case, our model achieved the top rank on a standard chest X-ray question-answering benchmark without access to any images. Third, when models were explicitly instructed to guess answers without image access, rather than being implicitly prompted to assume images were present, performance declined markedly. Explicit guessing appears to engage a more conservative response regime, in contrast to the mirage regime in which models behave as though images have been provided. These findings expose fundamental vulnerabilities in how visual-language models reason and are evaluated, pointing to an urgent need for private benchmarks that eliminate textual cues enabling non-visual inference, particularly in medical contexts where miscalibrated AI carries the greatest consequence. We introduce B-Clean as a principled solution for fair, vision-grounded evaluation of multimodal AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。