构建医学多模态推理数据集,提升模型真实临床决策能力
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models

- 基于真实医学文献构建45万条带推理链的图文问答数据
- 训练后视觉问答准确率提升20%,接近顶尖医疗模型水平
- 适合需要可解释性医疗AI的开发者与研究者使用
高风险临床场景下大型视觉语言模型(LVLMs)需基于视觉证据和临床知识进行推理,而不仅是给出正确答案。我们提出OpenMedReason,一个大规模、开源的多模态医学推理语料库,包含约45万张图像-问题-答案实例,其推理过程主要源自经过筛选的生物医学和人工撰写的科学论文。该数据集覆盖放射影像、显微图像、可见光照片、图表等多种医学视觉模态。我们配套推出OpenMedReason-Bench基准测试,支持对LVLM在感知、医学知识和推理逻辑三个维度的细粒度评估,实现超越最终答案准确率的诊断分析。实验表明,使用OpenMedReason训练可使视觉问答准确率相比基线模型平均提升20%,性能仅比最强可比规模医疗LVLM低4.2%。细粒度分析显示,该数据集同时提升了感知、知识和推理能力,其生成的推理链在86.1%的成对比较中优于基线模型。代码与数据集已公开于huggingface.co/datasets/neginb/OpenMedReason。
原文摘要 · Abstract (English)
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We introduce OpenMedReason, a large-scale, open multimodal medical reasoning corpus comprising approximately 450K image-question-answer instances whose reasoning traces are primarily derived from curated biomedical, human-authored scientific articles. OpenMedReason provides high-fidelity supervision beyond synthetic chains of thought, covering diverse medical domain vision modalities such as radiological scans, microscopic images, visible light photographs, charts, and others. We complement it with OpenMedReason-Bench, a held-out benchmark that allows fine-grained evaluation of LVLMs along three complementary axes of capability, including perception, medical knowledge, and rationale, enabling diagnostic evaluation beyond final-answer accuracy. OpenMedReason is a rich training resource that exhibits its effectiveness in both supervised fine-tuning (SFT) and reinforcement-based alignment. Training with OpenMedReason yields a 20% average improvement in VQA accuracy over the base model and achieves performance within 4.2% of the strongest comparable-scale medical LVLMs. Fine-grained performance analysis confirms that the gains are not concentrated in any single axis: OpenMedReason improves perception, medical knowledge, and rationale jointly, and its reasoning traces are preferred over those of the base model in 86.1% of pairwise comparisons. We release the code and dataset at huggingface.co/datasets/neginb/OpenMedReason.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。