让SAM自动生成提示，实现3D医学图像精准分割
Self-Prompt SAM: Medical Image Segmentation via Automatic Prompt SAM Adaptation
- 自动生成点、框提示，无需人工标注
- 3D深度融合适配器使2D模型支持3D医学图像
- 在多个数据集上超越nnUNet，最高提升2.3%
分割一切模型（SAM）在自然图像分割中展现出卓越的零样本性能，但其在医学图像分割中的表现仍不明确，因医学图像与自然图像差异显著。同时，SAM需人工提供点或框等额外提示，这对医学图像而言难以实现。本文提出一种自提示SAM适应框架Self-Prompt-SAM：设计多尺度提示生成器，结合SAM图像编码器生成辅助掩码；利用辅助掩码生成框提示，通过距离变换选取最中心点作为点提示。此外，设计3D深度融合适配器（DfusedAdapter），将其注入图像编码器和掩码解码器的每个Transformer层，使预训练2D SAM模型可提取3D信息并适应3D医学图像。大量实验表明，本方法在AMOS2022、ACDC和Synapse数据集上分别优于nnUNet 2.3%、1.6%和0.5%，达到当前最佳性能。
原文摘要 · Abstract (English)
Segment Anything Model (SAM) has demonstrated impressive zero-shot performance and brought a range of unexplored capabilities to natural image segmentation tasks. However, as a very important branch of image segmentation, the performance of SAM remains uncertain when applied to medical image segmentation due to the significant differences between natural images and medical images. Meanwhile, it is harsh to meet the SAM's requirements of extra prompts provided, such as points or boxes to specify medical regions. In this paper, we propose a novel self-prompt SAM adaptation framework for medical image segmentation, named Self-Prompt-SAM. We design a multi-scale prompt generator combined with the image encoder in SAM to generate auxiliary masks. Then, we use the auxiliary masks to generate bounding boxes as box prompts and use Distance Transform to select the most central points as point prompts. Meanwhile, we design a 3D depth-fused adapter (DfusedAdapter) and inject the DFusedAdapter into each transformer in the image encoder and mask decoder to enable pre-trained 2D SAM models to extract 3D information and adapt to 3D medical images. Extensive experiments demonstrate that our method achieves state-of-the-art performance and outperforms nnUNet by 2.3% on AMOS2022, 1.6% on ACDCand 0.5% on Synapse datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。