arXiv:2606.01044cs.CV2026-06

通过视觉扰动评估问题风险,减少医学图像问答中的幻觉答案。

Ask4VG: Risk-Aware Question Selection for Reducing Prior-Driven Answers in Medical VQA

论文配图:Ask4VG: Risk-Aware Question Selection for Reducing Prior-Driven Answers in Medical VQA
图 1 · 摘自论文原文
  • 用图像扰动生成反事实答案,评估问题引发幻觉的风险。
  • 风险重排序使准确率从33.7%提升至35.6%,风险值从0.658降至0.623。
  • 适合关注医疗问答可信度的研究者和临床AI应用开发者。

医学视觉问答要求模型基于图像证据生成答案,因为缺乏视觉支持的回答可能误导后续判断。然而,许多医学VQA问题形式模板化、高度相似,易导致模型学习问题-答案捷径而非依赖图像推理,增加幻觉风险。本文提出Ask4VG,一种无需标签的感知风险问题选择框架。通过反事实视觉探测:同一问题在原始图像、扰动图像、空白图像和错配图像上提问,根据答案差异构建弱监督信号,训练反事实风险估计器。该估计器对候选问题重写进行风险重排序,优先选择保留语义意图但对缺失或错位视觉信息不敏感的问题。在VQA-RAD数据集上,仅使用提示重写使反事实风险上升,而预测风险重排序将保留风险从0.658降至0.623,准确率由0.337提升至0.356。300样本的PMC-VQA外部验证也显示风险下降趋势与小幅准确率提升。结果表明,问题选择是应对幻觉的重要补充手段。

原文摘要 · Abstract (English)

Medical visual question answering requires models to ground their responses in image evidence, because visually unsupported answers can mislead downstream interpretation. However, many medical VQA questions are generic, template-like, or highly similar in form, which can encourage models to learn question-answer shortcuts instead of image-dependent reasoning and thereby increase the risk of hallucinated responses. We propose Ask4VG, a label-free pilot framework for risk-aware question selection. Ask4VG estimates question-induced hallucination risk through counterfactual visual probing: the same question is asked under the original image, a perturbed image, a blank image, and a mismatched image, and the resulting answer relations are converted into weak supervision for a counterfactual risk estimator. The learned estimator then reranks candidate question rewrites to favor intent-preserving questions that are less invariant to missing or mismatched visual evidence before final answer generation. On VQA-RAD with Qwen2-VL-2B-Instruct, prompt-only rewriting increases counterfactual risk, whereas predicted-risk reranking reduces held-out risk from 0.658 to 0.623 and improves exact accuracy from 0.337 to 0.356. A 300-sample PMC-VQA external check shows the same direction of risk reduction with a small accuracy gain. These results suggest that question selection is a promising complement to response-level hallucination mitigation for reliable medical VQA.

医学VQA幻觉检测反事实推理风险评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。