通过解剖相似性课程学习提升医学图像理解能力
Anatomy-R1: Enhancing Anatomy Reasoning in Multimodal Large Language Models via Anatomical Similarity Curriculum and Group Diversity Augmentation
- 按解剖结构相似性渐进设计题目难度，促进知识迁移
- 引入群体多样性问答增强，避免模型陷入单一推理路径
- 在SGG-VQA和OmniMedVQA上显著提升医学推理性能
多模态大语言模型在自然图像推理中取得显著进展，但在临床解剖手术图像领域潜力尚未充分挖掘。解剖理解任务需要精确且符合临床逻辑的回答，但受限于医学数据复杂性和高质量专家标注稀缺，传统监督微调策略效果有限。尽管近期研究显示组相对策略优化（GRPO）可在少数据下提升推理能力，我们发现其在解剖识别中存在两大缺陷：1）不同解剖结构间知识难以共享，导致信息获取不均，影响模型收敛；2）模型快速收敛至单一推理路径，抑制多样策略探索。为此，我们提出两项新方法：首先，采用解剖相似性课程学习（Anatomical Similarity Curriculum Learning），通过答案选项的相似度控制问题难度，实现复杂问题的渐进式掌握；其次，引入群体多样性问答增强（Group Diversity Question Augmentation），扩大困难样本的搜索空间，缓解回答同质化问题。在SGG-VQA和OmniMedVQA基准上的全面实验表明，本方法在两个基准上均有显著提升，验证了其在增强多模态大模型医学推理能力方面的有效性。代码已开源：https://github.com/tomato996/Anatomy-R1。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have achieved impressive progress in natural image reasoning, yet their potential in medical imaging remains underexplored, especially in clinical anatomical surgical images. Anatomy understanding tasks demand precise understanding and clinically coherent answers, which are difficult to achieve due to the complexity of medical data and the scarcity of high-quality expert annotations. These challenges limit the effectiveness of conventional Supervised Fine-Tuning (SFT) strategies. While recent work has demonstrated that Group Relative Policy Optimization (GRPO) can enhance reasoning in MLLMs without relying on large amounts of data, we find two weaknesses that hinder GRPO's reasoning performance in anatomy recognition: 1) knowledge cannot be effectively shared between different anatomical structures, resulting in uneven information gain and preventing the model from converging, and 2) the model quickly converges to a single reasoning path, suppressing the exploration of diverse strategies. To overcome these challenges, we propose two novel methods. First, we implement a progressive learning strategy called Anatomical Similarity Curriculum Learning by controlling question difficulty via the similarity of answer choices, enabling the model to master complex problems incrementally. Second, we utilize question augmentation referred to as Group Diversity Question Augmentation to expand the model's search space for difficult queries, mitigating the tendency to produce uniform responses. Comprehensive experiments on the SGG-VQA and OmniMedVQA benchmarks show our method achieves a significant improvement across the two benchmarks, demonstrating its effectiveness in enhancing the medical reasoning capabilities of MLLMs. The code can be found in https://github.com/tomato996/Anatomy-R1
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。