通用视觉模型在2D医学图像分割中表现优于多数专用模型。
Are General-Purpose Vision Models All We Need for 2D Medical Image Segmentation? A Cross-Dataset Empirical Study
- 统一训练评估协议下对比11种模型,涵盖专用与通用视觉模型。
- 在3个异构数据集上,通用模型整体分割精度更高。
- 无需专门设计,通用模型也能捕捉临床相关结构,解释性良好。
医学图像分割是计算机辅助诊断和临床决策支持系统的基础。过去十年中,为应对低对比度、小结构、标注数据少等医学影像特有问题,出现了众多专用分割架构。与此同时,计算机视觉领域发展出性能强大的通用视觉模型(GP-VMs),最初面向自然图像设计。尽管它们在标准视觉基准上表现优异,但在医学图像分割中的有效性仍不明确。本文开展受控实证研究,检验专用医学分割架构(SMAs)是否在2D MIS中系统性优于现代通用视觉模型。我们在统一训练与评估协议下,比较了11种SMAs与GP-VMs,在三个异构数据集上进行实验,覆盖不同成像模态、类别结构和数据特征。除了分割精度,还通过定性Grad-CAM可视化分析可解释性(XAI)。结果表明,在所分析数据集中,GP-VMs优于多数专用模型;且XAI分析显示,无需显式领域特定设计,GP-VMs仍能捕捉临床相关结构。这些发现表明,通用视觉模型可作为专用方法的可行替代,强调了端到端医学分割系统中明智模型选择的重要性。所有代码与资源已在GitHub公开。
原文摘要 · Abstract (English)
Medical image segmentation (MIS) is a fundamental component of computer-assisted diagnosis and clinical decision support systems. Over the past decade, numerous architectures specifically tailored to medical imaging have emerged to address domain-specific challenges such as low contrast, small anatomical structures, and limited annotated data. In parallel, rapid progress in computer vision has produced highly capable general-purpose vision models (GP-VMs) originally designed for natural images. Despite their strong performance on standard vision benchmarks, their effectiveness for MIS remains insufficiently understood. In this work, we conduct a controlled empirical study to examine whether specialized medical segmentation architectures (SMAs) provide systematic advantages over modern GP-VMs for 2D MIS. We compare eleven SMAs and GP-VMs using a unified training and evaluation protocol. Experiments are performed across three heterogeneous datasets covering different imaging modalities, class structures, and data characteristics. Beyond segmentation accuracy, we analyze qualitative Grad-CAM visualizations to investigate explainability (XAI) behavior. Our results demonstrate that, for the analyzed datasets, GP-VMs out-perform the majority of specialized MIS models. Moreover, XAI analyses indicate that GP-VMs can capture clinically relevant structures without explicit domain-specific architectural design. These findings suggest that GP-VMs can represent a viable alternative to domain-specific methods, highlighting the importance of informed model selection for end-to-end MIS systems. All code and resources are available at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。