用推理轨迹监督提升医学视觉问答模型的逻辑能力。
Improving Medical VQA through Trajectory-Aware Process Supervision
- 通过生成六大数据集的推理过程，引入轨迹感知奖励。
- 结合DTW距离与精确匹配，平均准确率从59.8%提至68.9%。
- 适合研究医疗视觉语言模型推理机制的学者使用。
可靠的医学视觉问答需要强大的推理能力，但现有数据集极少包含推理解释。本文利用COMCTS算法和开源视觉-语言模型生成六个医学VQA基准的推理轨迹，并由大语言模型作为验证裁判。基于这些数据，提出两阶段训练框架：先进行监督微调，再采用新型基于过程的奖励进行组相对策略优化（GRPO）。不同于仅依赖最终答案精确匹配的常规方法，我们设计了轨迹感知奖励，通过句向量嵌入与动态时间规整（DTW）计算生成与真实推理路径的相似性。在六个基准上的实验表明，结合DTW过程奖励与精确匹配奖励，显著优于仅使用SFT训练：平均准确率从0.598提升至0.689，平均BERTScore从0.845升至0.881，平均ROUGE-L从0.665增至0.748。结果强调了过程监督对训练具备推理能力的医学视觉语言模型的重要性。代码与生成数据集已公开于https://anonymous.4open.science/r/MICCAI-R1-MED-VQA-code-B14B/
原文摘要 · Abstract (English)
Reasoning capabilities are crucial for reliable medical visual question answering (VQA); however, existing datasets rarely include reasoning explanations. We address this by generating reasoning trajectories for six medical VQA benchmarks using the COMCTS algorithm with open-source vision-language models, with an LLM serving as the verification judge. Building on these generated datasets, we propose a two-stage training framework: supervised fine-tuning followed by Group Relative Policy Optimization (GRPO) with a novel process-based reward. While standard approaches rely solely on exact-match rewards for final answers, we introduce a trajectory-aware reward that measures the similarity between generated and ground-truth reasoning processes. Specifically, we embed reasoning steps using sentence transformers and compute the Dynamic Time Warping (DTW) distance between the resulting vector sequences. Experiments across six benchmarks demonstrate that combining the DTW-based process reward with exact-match reward consistently outperforms SFT-only training, raising mean accuracy from 0.598 to 0.689, mean BERTScore from 0.845 to 0.881, and mean ROUGE-L from 0.665 to 0.748. Our results highlight the importance of process supervision in training reasoning-capable medical VLMs. We make our code and generated reasoning datasets publicly available at https://anonymous.4open.science/r/MICCAI-R1-MED-VQA-code-B14B/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。