让AI像医生一样逐步聚焦病灶,提升医学影像诊断准确性。
MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis
- 通过动态注视机制引导模型分步分析医学图像区域。
- 在多个医疗视觉问答任务上平均提升8.5个百分点。
- 适合需要可解释性医疗AI的临床研究与系统开发。
准确的医学诊断常依赖逐步聚焦和迭代推理,这在临床流程中普遍存在。尽管近期视觉语言模型通过强化学习结合可验证奖励(RLVR)展现出出色的思维链(CoT)推理能力,但其纯在线学习范式容易强化表面连贯却临床错误的推理路径。我们提出MedEyes,一种新型强化学习框架,通过动态建模临床医生式的诊断推理过程,逐步关注并解析相关医学图像区域。通过引入离线专家指导,MedEyes将专家的视觉搜索轨迹转化为结构化的外部行为信号,引导模型走向临床一致的视觉推理。我们设计了凝视引导推理导航器(GRN),通过双模式探索策略实现系统性异常定位与深入局部分析。为平衡专家模仿与自主发现,引入置信度值采样器(CVS),采用核采样与自适应终止生成多样且可信的探索路径。最后,双流GRPO优化框架解耦在线与离线学习信号,缓解奖励同化与熵崩溃问题。实验表明,MedEyes在多个医疗视觉问答基准上平均性能提升+8.5个百分点,验证了其构建可信医疗AI系统的潜力。代码已开源于 https://github.com/zhcz328/MedEyes。
原文摘要 · Abstract (English)
Accurate medical diagnosis often involves progressive visual focusing and iterative reasoning, characteristics commonly observed in clinical workflows. While recent vision-language models demonstrate promising chain-of-thought (CoT) reasoning capabilities via reinforcement learning with verifiable rewards (RLVR), their purely on-policy learning paradigm tends to reinforce superficially coherent but clinically inaccurate reasoning paths. We propose MedEyes, a novel reinforcement learning framework that dynamically models clinician-style diagnostic reasoning by progressively attending to and interpreting relevant medical image regions. By incorporating off-policy expert guidance, MedEyes converts expert visual search trajectories into structured external behavioral signals, guiding the model toward clinically aligned visual reasoning. We design the Gaze-guided Reasoning Navigator (GRN) to emulate the diagnostic process through a dual-mode exploration strategy, scanning for systematic abnormality localization and drilling for detailed regional analysis. To balance expert imitation and autonomous discovery, we introduce the Confidence Value Sampler (CVS), which employs nucleus sampling and adaptive termination to create diverse yet credible exploration paths. Finally, the dual-stream GRPO optimization framework decouples on-policy and off-policy learning signals, mitigating reward assimilation and entropy collapse. Experiments demonstrate that MedEyes achieves an average performance improvement of +8.5pp across multiple medical VQA benchmarks, validating MedEyes's potential in building trustworthy medical AI systems. Code is available at https://github.com/zhcz328/MedEyes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。