首个医学大模型鲁棒性评测，揭示微调策略与医疗特有干扰的关键影响。
MedFM-Robust: Benchmarking Robustness of Medical Foundation Models
- 构建包含40种扰动的医学影像鲁棒性评测基准，覆盖8种成像模态。
- 医疗特异性扰动对分割任务影响显著，9个最严重干扰为领域专属类型。
- 零样本生成稳定，但LoRA微调导致视觉定位性能下降超40分，需谨慎选择策略。
医学基础模型虽在临床任务中表现优异，但其在真实世界扰动下的鲁棒性仍缺乏系统评估。本文构建了一个涵盖40种扰动类型（12种基础、28种医学特异性）的鲁棒性评测基准，覆盖八种成像模态，评估五种视觉语言模型（LLaVA-Med、MedGemma、MedGemma-1.5、Gemini-2.5-flash、GPT-4o-mini）在VQA、视觉定位和图像描述任务上的表现，以及两种分割模型（MedSAM、SAM-Med2D）搭配五种微调策略的结果。研究发现：(1) 微调策略显著影响鲁棒性，LoRA微调的性能退化接近全量微调的两倍；而SAM-Med2D的Adapter在效率与鲁棒性间取得良好平衡。(2) 医学特异性扰动对分割任务破坏力更强，15个最严重的图像损坏中有9个为领域相关。(3) LoRA微调后的视觉定位性能下降超过40点，而零样本图像描述保持稳定（降幅<7%）。零样本VQA表现因模型而异——医学模型性能下降低于20%，但Gemini-2.5-flash下降达54%。通用视觉语言模型在VQA上精度更高，但在视觉定位上表现不佳；在医学模型中，MedGemma展现出最佳整体稳定性。结果为医学AI部署提供指导，并强调开展领域特定鲁棒性评估的重要性。代码已开源：https://abnerai.github.io/MedFM-Robust。
原文摘要 · Abstract (English)
Medical foundation models have achieved remarkable clinical performance, yet their robustness under real-world perturbations remains underexplored. We present a robustness benchmark comprising 40 perturbation types (12 base, 28 medical-specific) across eight imaging modalities, evaluating five VLMs (LLaVA-Med, MedGemma, MedGemma-1.5, Gemini-2.5-flash and GPT-4o-mini) on VQA, visual grounding, and captioning, alongside two segmentation models (MedSAM, SAM-Med2D) with five fine-tuning strategies. Our findings reveal: (1) Fine-tuning strategy dominates robustness, with LoRA exhibiting nearly double the degradation of full fine-tuning, while SAM-Med2D's Adapter offers favorable efficiency-robustness trade-off. (2) Medical-specific perturbations disproportionately damage segmentation, with 9 of 15 top corruptions being domain-specific. (3) LoRA-tuned visual grounding drops over 40 points, whereas zero-shot captioning remains stable (<7% drop). Zero-shot VQA shows model-dependent robustness--medical models drop under 20% while Gemini-2.5-flash drops 54%. General-purpose VLMs achieve higher VQA accuracy but fail on grounding; among medical VLMs, MedGemma demonstrates the best overall stability. These results provide deployment guidelines and underscore the necessity of domain-specific robustness evaluation for medical AI. Our code is available at: https://abnerai.github.io/MedFM-Robust.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。