arXiv:2409.04298cs.CV2024-09被引 7

无需微调,用反向传播提升医学图像分割精度

RevSAM2: Prompt SAM2 for Medical Image Segmentation via Reverse-Propagation without Fine-tuning

  • 通过反向传播筛选高质量查询信息,优化提示选择机制
  • 在仅用少量标注样本下,Dice得分比现有方法高12.18%
  • 适合罕见病或新类别等标注数据稀缺的医学场景

Segment Anything Model 2(SAM2)在自然图像和视频的零样本提示分割中表现优异。然而,当将SAM2的传播机制用于医学图像时,常因空间不一致性导致相似图像分割结果差异显著。本文提出RevSAM2,一种无需微调的自校正框架,可实现对未见3D医学图像分割任务的卓越性能。针对仅用少量支持图像-标签对定义新任务的3D查询体,我们提出反向传播策略作为查询信息选择机制:不再简单使用先进先出队列逐片预测,而是利用支持图像评估每片预测掩码质量,筛选高质量掩码作为提示,跨整个查询体传播,从而增强对未见任务的泛化能力。首次探索了在无微调条件下利用SAM2进行标签高效医学图像分割的潜力。相比在大规模标注数据上微调,该方案为罕见病或未见类别分割提供低成本替代方案。在四个公开数据集上的实验表明,在标签有限场景下,RevSAM2性能优于当前最优方法,Dice得分提升12.18%。代码将公开。

原文摘要 · Abstract (English)

The Segment Anything Model 2 (SAM2) has recently demonstrated exceptional performance in zero-shot prompt segmentation for natural images and videos. However, when the propagation mechanism of SAM2 is applied to medical images, it often results in spatial inconsistencies, leading to significantly different segmentation outcomes for very similar images. In this paper, we introduce RevSAM2, a simple yet effective self-correction framework that enables SAM2 to achieve superior performance in unseen 3D medical image segmentation tasks without the need for fine-tuning. Specifically, to segment a 3D query volume using a limited number of support image-label pairs that define a new segmentation task, we propose reverse propagation strategy as a query information selection mechanism. Instead of simply maintaining a first-in-first-out (FIFO) queue of memories to predict query slices sequentially, reverse propagation selects high-quality query information by leveraging support images to evaluate the quality of each predicted query slice mask. The selected high-quality masks are then used as prompts to propagate across the entire query volume, thereby enhancing generalization to unseen tasks. Notably, we are the first to explore the potential of SAM2 in label-efficient medical image segmentation without fine-tuning. Compared to fine-tuning on large labeled datasets, the label-efficient scenario provides a cost-effective alternative for medical segmentation tasks, particularly for rare diseases or when dealing with unseen classes. Experiments on four public datasets demonstrate the superiority of RevSAM2 in scenarios with limited labels, surpassing state-of-the-arts by 12.18% in Dice. The code will be released.

医学图像分割零样本自校正标签效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。