用少量数据训练医疗影像模型，发现强化学习让模型更擅长基准测试却更难跨机构应用。
Benchmark Success, Clinical Failure: When Reinforcement Learning Optimizes for Benchmarks, Not Patients
- 基于2000样本微调+1000样本强化学习，在单块A100上训练视觉语言模型
- 模型在本地数据集提升23%准确率但跨数据集性能下降19%，通用性变差
- 医疗预训练模型中，强化学习收益有限，监督微调更适合临床部署
近期大语言模型的强化学习进展提升了推理能力，但在资源受限的医学影像领域仍缺乏探索。我们提出ChexReason，采用SFT后接GRPO的R1风格方法，仅用2000个SFT样本、1000个RL样本和一块A100 GPU训练。在CheXpert和NIH数据集上的评估显示：GRPO虽恢复了分布内性能（CheXpert宏平均F1达0.346，提升23%），但显著降低跨数据集泛化能力（NIH性能下降19%）。这一现象与高资源模型NV-Reason-CXR-3B一致，表明问题源于强化学习范式本身而非规模。我们发现一种泛化悖论——初始SFT检查点在NIH上表现优于优化后模型，说明教师引导的推理更捕捉机构无关特征。此外，结构化推理框架对通用视觉语言模型有帮助，但对医疗预训练模型提升甚微。因此，精心设计的监督微调可能比激进的强化学习更适合需要跨人群鲁棒性的临床部署。
原文摘要 · Abstract (English)
Recent Reinforcement Learning (RL) advances for Large Language Models (LLMs) have improved reasoning tasks, yet their resource-constrained application to medical imaging remains underexplored. We introduce ChexReason, a vision-language model trained via R1-style methodology (SFT followed by GRPO) using only 2,000 SFT samples, 1,000 RL samples, and a single A100 GPU. Evaluations on CheXpert and NIH benchmarks reveal a fundamental tension: GRPO recovers in-distribution performance (23% improvement on CheXpert, macro-F1 = 0.346) but degrades cross-dataset transferability (19% drop on NIH). This mirrors high-resource models like NV-Reason-CXR-3B, suggesting the issue stems from the RL paradigm rather than scale. We identify a generalization paradox where the SFT checkpoint uniquely improves on NIH before optimization, indicating teacher-guided reasoning captures more institution-agnostic features. Furthermore, cross-model comparisons show structured reasoning scaffolds benefit general-purpose VLMs but offer minimal gain for medically pre-trained models. Consequently, curated supervised fine-tuning may outperform aggressive RL for clinical deployment requiring robustness across diverse populations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。