用Mamba增强SAM,实现无需训练的肠息肉精准分割
SAM-Mamba: Mamba Guided SAM Architecture for Generalized Zero-Shot Polyp Segmentation
- 在SAM编码器中加入Mamba-Prior模块,注入息肉特有线索
- 在5个数据集上超越CNN、ViT等模型,零样本泛化能力强
- 适合临床实时应用,对未见数据适应性好
结肠镜下息肉分割对早期发现结直肠癌至关重要。由于息肉形态、颜色和大小差异大,且与周围组织边界模糊,分割难度高。传统基于卷积神经网络(CNN)的模型难以捕捉细节和全局上下文,性能受限;基于视觉变压器(ViT)的模型虽能处理部分问题,但对局部上下文建模不足,且零样本泛化能力弱。为此,我们提出Mamba引导的通用零样本息肉分割模型SAM-Mamba。该方法在编码器中引入Mamba-Prior模块,将息肉图像的关键特征作为领域先验注入SAM的图像编码器,同时在多尺度下捕获全局依赖关系,显著提升分割精度。在五个基准数据集上的实验表明,SAM-Mamba在定量与定性指标上均优于传统CNN、ViT及适配器类模型。此外,其对未见数据具有优异适应性,适用于实时临床部署。
原文摘要 · Abstract (English)
Polyp segmentation in colonoscopy is crucial for detecting colorectal cancer. However, it is challenging due to variations in the structure, color, and size of polyps, as well as the lack of clear boundaries with surrounding tissues. Traditional segmentation models based on Convolutional Neural Networks (CNNs) struggle to capture detailed patterns and global context, limiting their performance. Vision Transformer (ViT)-based models address some of these issues but have difficulties in capturing local context and lack strong zero-shot generalization. To this end, we propose the Mamba-guided Segment Anything Model (SAM-Mamba) for efficient polyp segmentation. Our approach introduces a Mamba-Prior module in the encoder to bridge the gap between the general pre-trained representation of SAM and polyp-relevant trivial clues. It injects salient cues of polyp images into the SAM image encoder as a domain prior while capturing global dependencies at various scales, leading to more accurate segmentation results. Extensive experiments on five benchmark datasets show that SAM-Mamba outperforms traditional CNN, ViT, and Adapter-based models in both quantitative and qualitative measures. Additionally, SAM-Mamba demonstrates excellent adaptability to unseen datasets, making it highly suitable for real-time clinical use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。