arXiv:2505.17096eess.IVcs.CV2025-05ICCV被引 1

用多提示融合让2D模型精准分割3D肿瘤,性能超越现有方法46.88%。

TAGS: 3D Tumor-Adaptive Guidance for SAM

  • 通过多提示融合保留预训练权重,结合语义与解剖提示增强特征提取
  • 在3个公开数据集上分割精度领先nnUNet 46.88%,其他模型至少提升13%
  • 适合需快速适配2D大模型到3D医学图像任务的研究者与临床应用

CLIP和SAM等基础模型在图像分割中表现优异,但其在3D医学影像(如病理检测与分割)中的应用仍不充分。主要挑战在于自然图像与医学体数据之间的领域差异:现有模型基于2D预训练,难以捕捉3D解剖结构上下文,限制了其在肿瘤分割等临床场景的应用。为此,我们提出名为TAGS(Tumor Adaptive Guidance for SAM)的适配框架,通过多提示融合,使2D基础模型适用于3D医学任务。该方法保留大部分预训练权重,利用CLIP的语义信息和解剖特定提示增强SAM的空间特征提取能力。在三个开源肿瘤分割数据集上的大量实验表明,本模型性能显著优于当前最先进的医学图像分割模型(较nnUNet提升46.88%)、交互式分割框架及其他主流医学基础模型(如SAM-Med2D、SAM-Med3D、SegVol、Universal、3D-Adapter、SAM-B),所有对比模型均至少提升13%。结果凸显了该框架在多样化医学分割任务中的鲁棒性与可扩展性。

原文摘要 · Abstract (English)

Foundation models (FMs) such as CLIP and SAM have recently shown great promise in image segmentation tasks, yet their adaptation to 3D medical imaging-particularly for pathology detection and segmentation-remains underexplored. A critical challenge arises from the domain gap between natural images and medical volumes: existing FMs, pre-trained on 2D data, struggle to capture 3D anatomical context, limiting their utility in clinical applications like tumor segmentation. To address this, we propose an adaptation framework called TAGS: Tumor Adaptive Guidance for SAM, which unlocks 2D FMs for 3D medical tasks through multi-prompt fusion. By preserving most of the pre-trained weights, our approach enhances SAM's spatial feature extraction using CLIP's semantic insights and anatomy-specific prompts. Extensive experiments on three open-source tumor segmentation datasets prove that our model surpasses the state-of-the-art medical image segmentation models (+46.88% over nnUNet), interactive segmentation frameworks, and other established medical FMs, including SAM-Med2D, SAM-Med3D, SegVol, Universal, 3D-Adapter, and SAM-B (at least +13% over them). This highlights the robustness and adaptability of our proposed framework across diverse medical segmentation tasks.

3D分割肿瘤分割基础模型多提示融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。