融合医学与抗退化模型,提升医疗图像分割在模糊噪声下的准确性
RobustMedSAM: Degradation-Resilient Medical Image Segmentation via Robust Foundation Model Adaptation

- 分模块融合医学专用编码器与抗退化解码器,协同提升鲁棒性
- 在35个医疗数据集上微调解码器,退化图像Dice提升0.106(0.613→0.719)
- 适合需要高可靠性医疗分割的临床场景,尤其应对真实世界图像质量问题
基于Segment Anything Model(SAM)的医疗图像分割模型在干净基准上表现优异,但在现实中的噪声、模糊、运动伪影及模态特异性失真下可靠性下降。现有方法仅关注医学领域适配或退化鲁棒性,未兼顾二者。我们发现:在SAM中,医学先验信息集中于图像编码器,而抗退化能力主要由掩码解码器决定。据此提出RobustMedSAM,采用模块化检查点融合策略——以MedSAM初始化图像编码器,以RobustSAM初始化掩码解码器,共享ViT-B结构。随后仅在包含六种成像模态和十二类退化的35个医疗数据集(MedSegBench)上微调掩码解码器,其余组件冻结以保留预训练医学表征。另探索基于SVD的参数高效变体以实现有限编码器适配。在分布内与分布外基准上实验表明,相比SAM,RobustMedSAM将退化图像的Dice分数从0.613提升至0.719(+0.106),验证了互补预训练模型的结构化融合是提升医疗图像分割鲁棒性的有效且实用路径。
原文摘要 · Abstract (English)
Medical image segmentation models built on Segment Anything Model (SAM) achieve strong performance on clean benchmarks, yet their reliability often degrades under realistic image corruptions such as noise, blur, motion artifacts, and modality-specific distortions. Existing approaches address either medical-domain adaptation or corruption robustness, but not both jointly. In SAM, we find that these capabilities are concentrated in complementary modules: the image encoder preserves medical priors, while the mask decoder governs corruption robustness. Motivated by this observation, we propose RobustMedSAM, which adopts module-wise checkpoint fusion by initializing the image encoder from MedSAM and the mask decoder from RobustSAM under a shared ViT-B architecture. We then fine-tune only the mask decoder on 35 medical datasets from MedSegBench, spanning six imaging modalities and 12 corruption types, while freezing the remaining components to preserve pretrained medical representations. We additionally investigate an SVD-based parameter-efficient variant for limited encoder adaptation. Experiments on both in-distribution and out-of-distribution benchmarks show that RobustMedSAM improves degraded-image Dice from 0.613 to 0.719 (+0.106) over SAM, demonstrating that structured fusion of complementary pretrained models is an effective and practical approach for robust medical image segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。