用2D医学图像训练的推理接口,提升3D医学视觉问答性能
UniReason-Med: A Shared Grounded Reasoning Interface for 2D-to-3D Transfer in Medical VQA

- 构建共享推理接口,统一处理2D图像与3D切片序列
- 在220K数据上训练后,3D任务准确率显著优于纯3D训练
- 适合需要跨模态医学影像理解的研究者
我们研究是否可通过丰富2D医学图像中的基于视觉的推理监督,提升在共享推理接口下对3D医学视觉问答(VQA)的表现。提出UniReason-Med,一个单检查点框架,可在推理时处理2D图像或序列化3D体积,通过共享框语法、区域标记注入和统一的基于视觉的推理策略,生成交错的文本推理与定位视觉证据。为训练该接口,构建了包含22万条指令微调样本的UniMed-CoT数据集,其中17万为2D样本,5万为3D样本。通过监督微调结合结果级强化学习,UniReason-Med在强化学习阶段不依赖IoU/Dice等定位奖励,仍能生成准确的推理轨迹。数据混合与组件消融实验表明,联合2D+3D的基于视觉监督显著提升3D推理性能,而基于视觉的对齐与区域标记注入对2D和3D任务均有持续收益。结果表明,共享的基于视觉推理接口可将2D图像的推理结构迁移至3D切片序列理解。代码与数据已公开于https://github.com/IQuestLab/unireason-med。
原文摘要 · Abstract (English)
We study whether grounded reasoning supervision from abundant 2D medical images can improve 3D medical VQA when both input types are aligned through a common reasoning interface. We introduce UniReason-Med, a single-checkpoint framework that processes either a 2D image or a slice-serialized 3D volume at inference time, generating interleaved textual reasoning and localized visual evidence through shared box syntax, region-token injection, and a common grounded reasoning policy. To train this interface, we construct UniMed-CoT, a 220K instruction-tuning dataset with interleaved textual reasoning and grounded visual evidence, including 170K 2D and 50K 3D samples. Through supervised fine-tuning followed by outcome-level reinforcement learning, UniReason-Med learns to generate grounded reasoning traces without IoU/Dice-based localization rewards during RL. Data-mixture and component ablations show that joint 2D+3D grounded supervision substantially improves 3D reasoning over 3D-only training, while grounding and region-token injection consistently benefit both 2D and 3D tasks. These results suggest that a shared grounded reasoning interface can transfer reasoning structure from 2D images to slice-serialized volumetric medical understanding. The code and data are publicly available at https://github.com/IQuestLab/unireason-med.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。