让医学超声图像自动分割无需人工提示，性能提升最高达33%
Sam2Rad: A Segmentation Model for Medical Images with Learnable Prompts
- 用可学习提示网络自动生成分割所需提示
- 在三个骨骼超声数据集上提升2-33%的分割精度
- 仅需10张标注图即可训练，适配所有SAM结构
像通用分割模型（如SAM）这类基础模型需要高质量的人工提示进行医学图像分割，耗时且依赖专业经验。现有SAM及其变体在超声（US）图像中因领域差异常表现不佳。我们提出Sam2Rad，一种通过可学习提示适配SAM及其变体进行超声骨结构自动分割的方法。该方法引入提示预测网络（PPN），利用交叉注意力模块从图像编码器特征中生成提示嵌入，输出边界框、掩码提示及256维兴趣区域嵌入。框架支持可选人工提示，可通过参数高效微调（PEFT）端到端训练。在3个肌骨超声数据集上测试：腕部（3822张）、肩袖（1605张）、髋部（4849张）。无手动提示条件下，所有数据集性能均提升，髋部/腕部Dice分数提高2-7%，肩部数据最高提升33%。模型可仅用10张标注图像训练，兼容任意SAM架构实现全自动分割。
原文摘要 · Abstract (English)
Foundation models like the segment anything model require high-quality manual prompts for medical image segmentation, which is time-consuming and requires expertise. SAM and its variants often fail to segment structures in ultrasound (US) images due to domain shift. We propose Sam2Rad, a prompt learning approach to adapt SAM and its variants for US bone segmentation without human prompts. It introduces a prompt predictor network (PPN) with a cross-attention module to predict prompt embeddings from image encoder features. PPN outputs bounding box and mask prompts, and 256-dimensional embeddings for regions of interest. The framework allows optional manual prompting and can be trained end-to-end using parameter-efficient fine-tuning (PEFT). Sam2Rad was tested on 3 musculoskeletal US datasets: wrist (3822 images), rotator cuff (1605 images), and hip (4849 images). It improved performance across all datasets without manual prompts, increasing Dice scores by 2-7% for hip/wrist and up to 33% for shoulder data. Sam2Rad can be trained with as few as 10 labeled images and is compatible with any SAM architecture for automatic segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。