arXiv:2601.18240cs.CV2026-01被引 2

通过视觉逻辑回路验证,精准识别医学图像问答中的幻觉

V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answering

  • 构建双向推理回路,用答案反推问题验证视觉依据
  • 在多个医疗VQA数据集上超越现有方法,准确率显著提升
  • 无需训练可即插即用,适合临床诊断系统增强可信度

多模态大语言模型在医学视觉问答中展现强大诊断辅助能力,但其输出仍易产生与视觉事实矛盾的幻觉,危及高风险医疗场景。现有基于不确定性的检测方法虽高效,但仅间接估计预测不确定性,无法直接验证答案真实性。为此,我们提出无需训练、可即插即用的视觉逻辑回路验证(V-Loop)框架。V-Loop通过双向推理形成视觉锚定的逻辑闭环:给定输入后,模型生成主问答对的答案;V-Loop从中提取语义单元,以答案为条件生成验证问题,重新查询原问题单元,并强制视觉注意力一致性,确保主问题与验证问题依赖相同图像证据。若验证答案匹配预期语义内容,则逻辑回路闭合,表明答案具备事实依据;否则判定为主答案存在幻觉。在多个医学VQA基准和多款MLLM上的实验表明,V-Loop持续优于现有内省式方法,计算效率高,且与不确定性方法结合使用时进一步提升性能。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) have shown remarkable capability in assisting disease diagnosis in medical visual question answering (VQA). However, their outputs remain vulnerable to hallucinations (i.e., responses that contradict visual facts), posing significant risks in high-stakes medical scenarios. Recent introspective detection methods, particularly uncertainty-based approaches, offer computational efficiency but are fundamentally indirect, as they estimate predictive uncertainty for an image-question pair rather than verifying the factual correctness of a specific answer. To address this limitation, we propose Visual Logical Loop Verification (V-Loop), a training-free and plug-and-play framework for hallucination detection in medical VQA. V-Loop introduces a bidirectional reasoning process that forms a visually grounded logical loop to verify factual correctness. Given an input, the MLLM produces an answer for the primary input pair. V-Loop extracts semantic units from the primary QA pair, generates a verification question by conditioning on the answer unit to re-query the question unit, and enforces visual attention consistency to ensure answering both primary question and verification question rely on the same image evidence. If the verification answer matches the expected semantic content, the logical loop closes, indicating factual grounding; otherwise, the primary answer is flagged as hallucinated. Extensive experiments on multiple medical VQA benchmarks and MLLMs show that V-Loop consistently outperforms existing introspective methods, remains highly efficient, and further boosts uncertainty-based approaches when used in combination.

医疗AI幻觉检测视觉问答逻辑验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。