arXiv:2507.02994cs.LGcs.CV2025-07被引 21

无需复杂标注,用新奖励机制让医学图像定位模型更准

MedGround-R1: Advancing Medical Image Grounding via Spatial-Semantic Rewarded Group Relative Policy Optimization

  • 用空间语义奖励替代传统思维链标注,提升模型定位精度
  • 在三个医学数据集上达到当前最优性能,最高准确率超80%
  • 适合医疗AI研究者和需要精准图像理解的临床应用

医学图像定位(MIG)需根据文本描述定位医学影像中的特定区域,要求模型不仅感知区域,还需推断其空间关系。现有视觉语言模型依赖大量思维链(CoT)标注进行监督微调,成本高昂。本文将深度求索-1(DeepSeek-R1)提出的组相对策略优化(GRPO)框架适配至医学图像定位任务,提出空间-语义奖励的组相对策略优化方法,无需CoT标注即可训练模型。通过引入空间准确性奖励与语义一致性奖励,提供对正负完成结果的精细化反馈;并设计链式框(Chain-of-Box)模板,将参考框的视觉信息融入推理过程,使模型在中间步骤显式推理空间区域。在MS-CXR、ChestX-ray8和M3D-RefSeg三个数据集上的实验表明,该方法达到当前最优性能。消融实验证明各组件有效性。代码、模型权重及数据集已开源。

原文摘要 · Abstract (English)

Medical Image Grounding (MIG), which involves localizing specific regions in medical images based on textual descriptions, requires models to not only perceive regions but also deduce spatial relationships of these regions. Existing Vision-Language Models (VLMs) for MIG often rely on Supervised Fine-Tuning (SFT) with large amounts of Chain-of-Thought (CoT) reasoning annotations, which are expensive and time-consuming to acquire. Recently, DeepSeek-R1 demonstrated that Large Language Models (LLMs) can acquire reasoning abilities through Group Relative Policy Optimization (GRPO) without requiring CoT annotations. In this paper, we adapt the GRPO reinforcement learning framework to VLMs for Medical Image Grounding. We propose the Spatial-Semantic Rewarded Group Relative Policy Optimization to train the model without CoT reasoning annotations. Specifically, we introduce Spatial-Semantic Rewards, which combine spatial accuracy reward and semantic consistency reward to provide nuanced feedback for both spatially positive and negative completions. Additionally, we propose to use the Chain-of-Box template, which integrates visual information of referring bounding boxes into the <think> reasoning process, enabling the model to explicitly reason about spatial regions during intermediate steps. Experiments on three datasets MS-CXR, ChestX-ray8, and M3D-RefSeg demonstrate that our method achieves state-of-the-art performance in Medical Image Grounding. Ablation studies further validate the effectiveness of each component in our approach. Code, checkpoints, and datasets are available at https://github.com/bio-mlhui/MedGround-R1

医学图像视觉语言模型强化学习定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。