用弱监督提升医学图像分割，降低标注成本
WeakMedSAM: Weakly-Supervised Medical Image Segmentation via SAM with Sub-Class Exploration and Prompt Affinity Mining
- 引入子类探索模块缓解医学图像中类别共现问题
- 通过提示亲和性挖掘提升激活图质量，实现更准分割
- 适配SAM类模型，适合标注数据稀缺的医学场景
近年来，视觉领域基础模型取得显著进展。已有研究利用分割一切模型（SAM）提升医学图像分割性能，但多数方法依赖大量像素级标注数据进行全监督微调。为降低标注成本，本文提出一种新的弱监督SAM-based分割模型WeakMedSAM。该模型包含两个模块：1）子类探索模块用于缓解医学图像中的严重共现问题，学习更准确的特征表示；2）提示亲和性挖掘模块利用SAM的提示能力生成亲和图，用于随机游走优化。本方法可适配任意SAM-like骨干网络，实验在BraTS 2019、AbdomenCT-1K和MSD Cardiac三个主流数据集上验证了其有效性。代码已开源。
原文摘要 · Abstract (English)
We have witnessed remarkable progress in foundation models in vision tasks. Currently, several recent works have utilized the segmenting anything model (SAM) to boost the segmentation performance in medical images, where most of them focus on training an adaptor for fine-tuning a large amount of pixel-wise annotated medical images following a fully supervised manner. In this paper, to reduce the labeling cost, we investigate a novel weakly-supervised SAM-based segmentation model, namely WeakMedSAM. Specifically, our proposed WeakMedSAM contains two modules: 1) to mitigate severe co-occurrence in medical images, a sub-class exploration module is introduced to learn accurate feature representations. 2) to improve the quality of the class activation maps, our prompt affinity mining module utilizes the prompt capability of SAM to obtain an affinity map for random-walk refinement. Our method can be applied to any SAM-like backbone, and we conduct experiments with SAMUS and EfficientSAM. The experimental results on three popularly-used benchmark datasets, i.e., BraTS 2019, AbdomenCT-1K, and MSD Cardiac dataset, show the promising results of our proposed WeakMedSAM. Our code is available at https://github.com/wanghr64/WeakMedSAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。