通过分层自监督学习提升SAM在医学图像分割中的性能。
SAMora: Enhancing SAM through Hierarchical Self-Supervised Pre-Training for Medical Images
- 在图像、图块和像素层级应用互补自监督任务捕捉医学数据层次信息。
- 在少样本和全监督设置下均达到顶尖水平，微调轮次减少90%。
- 适配多种SAM变体，适用于医疗影像分割研究者。
分割一切模型（SAM）在医学图像分割中展现出巨大潜力，但在标注数据极少时表现受限，而医学数据中存在大量未被充分利用的层次信息。为此，我们受自监督学习启发，提出SAMora框架，通过在图像、图块和像素层级应用互补的自监督学习目标，捕获医学数据的层次知识。为充分挖掘LoRAs中多层次特征的互补性，引入HL-Attn分层融合模块，在保留各尺度特征特性的前提下实现多尺度特征融合。SAMora兼容多种SAM变体，包括SAM2、SAMed和H-SAM。在Synapse、LA和PROMISE12数据集上的实验表明，SAMora在少样本与全监督设置下均优于现有SAM变体，实现当前最佳性能，同时将微调轮次减少90%。代码已公开于https://github.com/ShChen233/SAMora。
原文摘要 · Abstract (English)
The Segment Anything Model (SAM) has demonstrated significant potential in medical image segmentation. Yet, its performance is limited when only a small amount of labeled data is available, while there is abundant valuable yet often overlooked hierarchical information in medical data. To address this limitation, we draw inspiration from self-supervised learning and propose SAMora, an innovative framework that captures hierarchical medical knowledge by applying complementary self-supervised learning objectives at the image, patch, and pixel levels. To fully exploit the complementarity of hierarchical knowledge within LoRAs, we introduce HL-Attn, a hierarchical fusion module that integrates multi-scale features while maintaining their distinct characteristics. SAMora is compatible with various SAM variants, including SAM2, SAMed, and H-SAM. Experimental results on the Synapse, LA, and PROMISE12 datasets demonstrate that SAMora outperforms existing SAM variants. It achieves state-of-the-art performance in both few-shot and fully supervised settings while reducing fine-tuning epochs by 90%. The code is available at https://github.com/ShChen233/SAMora.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。