用少量标注数据让SAM模型精准分割医学图像,无需大量训练
SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting
- 用聚类选代表样本,通过形变场传播掩码知识生成粗略分割
- 自动生成点、框、粗掩码作为提示,输入SAM获得高精度结果
- 仅需10个标注样例即可达到94.36%准确率,适合医疗数据少场景
医学图像分割常面临高昂的标注成本。尽管少样本学习可缓解此问题,传统方法仍依赖大量已知类别标注数据进行预训练。为此,我们提出利用在超过10亿个掩码上预训练的Segment Anything Model(SAM),避免对特定领域标注数据的依赖。在此基础上,我们构建了基于掩码传播与自动提示的SAM-MPA框架。首先,采用k-中心聚类选取最具代表性的样本构建支持集;随后,将这些标注样本配准至其他图像,生成形变场以传播掩码知识,获得全数据集的粗略掩码;接着,基于粗掩码的区域与边界扩展,自动生成包含点、框和粗掩码的视觉提示;最后,将提示输入SAM生成分割结果,并通过后处理模块优化。我们在两种不同模态的医学图像数据集上进行了大量实验,结果显示,在乳腺超声和胸部X光图像上分别达到74.53%和94.36%的Dice系数。实验证明,SAM-MPA仅需10个标注样本即可实现高精度分割,优于现有主流少样本自动分割方法。该方法可快速适配任意医学图像数据集,仅需少量标注。
原文摘要 · Abstract (English)
Medical image segmentation often faces the challenge of prohibitively expensive annotation costs. While few-shot learning offers a promising solution to alleviate this burden, conventional approaches still rely heavily on pre-training with large volumes of labeled data from known categories. To address this issue, we propose leveraging the Segment Anything Model (SAM), pre-trained on over 1 billion masks, thus circumventing the need for extensive domain-specific annotated data. In light of this, we developed SAM-MPA, an innovative SAM-based framework for few-shot medical image segmentation using Mask Propagation-based Auto-prompting. Initially, we employ k-centroid clustering to select the most representative examples for labelling to construct the support set. These annotated examples are registered to other images yielding deformation fields that facilitate the propagation of the mask knowledge to obtain coarse masks across the dataset. Subsequently, we automatically generate visual prompts based on the region and boundary expansion of the coarse mask, including points, box and a coarse mask. Finally, we can obtain the segmentation predictions by inputting these prompts into SAM and refine the results by post refinement module. We validate the performance of the proposed framework through extensive experiments conducted on two medical image datasets with different modalities. Our method achieves Dices of 74.53%, 94.36% on Breast US, Chest X-ray, respectively. Experimental results substantiate that SAM-MPA yields high-accuracy segmentations within 10 labeled examples, outperforming other state-of-the-art few-shot auto-segmentation methods. Our method enables the customization of SAM for any medical image dataset with a small number of labeled examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。