让医学视觉问答更可靠,通过动态保留关键视觉证据
MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering

- 在解码时插入连续潜变量,动态更新关键视觉信息
- 在多个医学数据集上将准确率从48.3%提升至53.4%
- 适合医疗AI诊断、可解释性研究者使用
医学视觉-语言模型在医学视觉问答任务中展现出巨大潜力,但其推理仍以文本为中心:图像仅被编码一次作为静态上下文,后续推理主要依赖语言。这一范式在临床场景中存在根本局限,因准确答案常依赖于细微且局部的视觉证据,而这些信息难以在静态嵌入中可靠保留。本文提出 extsc{MedLVR},一种隐式视觉推理框架,通过在自回归解码中引入显式的视觉证据状态,实现对查询相关视觉证据的迭代保持与优化。该方法通过重用隐藏状态作为连续潜变量步骤,在生成答案前持续精炼视觉证据。为支持有效视觉监督,采用两阶段训练策略:区域兴趣(ROI)监督微调使潜变量对齐临床相关图像证据,视觉潜变量策略优化(VLPO)则在结果级奖励下进一步优化潜变量推理与答案生成。在 OmniMedVQA 及五个外部医学 VQA 基准上的实验表明, extsc{MedLVR} 持续优于近期推理基线,将基于 Qwen2.5-VL-7B 骨干模型的平均得分从 48.3% 提升至 53.4%。结果表明,潜式视觉推理能有效保留诊断相关视觉证据,显著提升医学 VQA 的可靠性。
原文摘要 · Abstract (English)
Medical vision--language models (VLMs) have shown strong potential for medical visual question answering (VQA), yet their reasoning remains largely text-centric: images are encoded once as static context, and subsequent inference is dominated by language. This paradigm is fundamentally limited in clinical scenarios, where accurate answers often depend on subtle, localized visual evidence that cannot be reliably preserved in static embeddings. We propose \textsc{MedLVR}, a latent visual reasoning framework that introduces an explicit visual evidence state into autoregressive decoding. Instead of relying solely on text-based intermediate reasoning, \textsc{MedLVR} interleaves a short latent reasoning segment within the decoder by reusing hidden states as continuous latent steps, enabling iterative preservation and refinement of query-relevant visual evidence before answer generation. To support effective visual supervision, we adopt a two-stage training strategy: region of interest (ROI)-supervised fine-tuning aligns latent states with clinically relevant image evidence, and Visual-Latent Policy Optimization (VLPO) further optimizes latent reasoning and answer generation under outcome-level rewards. Experiments on OmniMedVQA and five external medical VQA benchmarks show that \textsc{MedLVR} consistently outperforms recent reasoning baselines and improves the average score over the Qwen2.5-VL-7B backbone from 48.3\% to 53.4\%. These results show that latent visual reasoning provides an effective mechanism for preserving diagnostically relevant visual evidence and improving the reliability of medical VQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。