用可学习提示增强SAM，实现医疗图像分割的高效半监督学习
Learnable Prompting SAM-induced Knowledge Distillation for Semi-supervised Medical Image Segmentation
- 设计可学习提示与双子网协同训练，动态生成密集提示并优化SAM适配
- 通过知识蒸馏使模型从SAM中学习，显著提升标注效率和分割精度
- 框架通用性强，可嵌入其他方法提升性能，适合医疗图像分割研究者
标注数据稀缺推动了半监督学习在医疗图像分割中的发展。针对通用分割大模型（如Segment Anything Model, SAM）在医疗领域表现下降的问题，本文提出一种可学习提示的SAM知识蒸馏框架（KnowSAM）。首先，采用多视图协同训练（MC）策略，利用两个不同子网络实现协同教学，提升鲁棒性；其次，设计可学习提示策略（LPS），动态生成密集提示，并引入适配器对SAM进行医学图像任务微调；此外，提出SAM诱导的知识蒸馏（SKD），将SAM的预测知识传递给两个子网络，缓解伪标签错误带来的影响。特别地，子网络生成的预测用于构造掩码提示输入SAM，促进模块间信息交互。在多个医疗分割任务上的实验表明，该方法优于现有最优半监督分割方法。关键的是，该知识蒸馏框架可无缝集成到其他半监督分割方法中以进一步提升性能。代码将在论文接收后公开于：https://github.com/taozh2017/KnowSAM
原文摘要 · Abstract (English)
The limited availability of labeled data has driven advancements in semi-supervised learning for medical image segmentation. Modern large-scale models tailored for general segmentation, such as the Segment Anything Model (SAM), have revealed robust generalization capabilities. However, applying these models directly to medical image segmentation still exposes performance degradation. In this paper, we propose a learnable prompting SAM-induced Knowledge distillation framework (KnowSAM) for semi-supervised medical image segmentation. Firstly, we propose a Multi-view Co-training (MC) strategy that employs two distinct sub-networks to employ a co-teaching paradigm, resulting in more robust outcomes. Secondly, we present a Learnable Prompt Strategy (LPS) to dynamically produce dense prompts and integrate an adapter to fine-tune SAM specifically for medical image segmentation tasks. Moreover, we propose SAM-induced Knowledge Distillation (SKD) to transfer useful knowledge from SAM to two sub-networks, enabling them to learn from SAM's predictions and alleviate the effects of incorrect pseudo-labels during training. Notably, the predictions generated by our subnets are used to produce mask prompts for SAM, facilitating effective inter-module information exchange. Extensive experimental results on various medical segmentation tasks demonstrate that our model outperforms the state-of-the-art semi-supervised segmentation approaches. Crucially, our SAM distillation framework can be seamlessly integrated into other semi-supervised segmentation methods to enhance performance. The code will be released upon acceptance of this manuscript at: https://github.com/taozh2017/KnowSAM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。