arXiv:2409.08492eess.IVcs.CV2024-09被引 18

用轻量设计让SAM高效分割3D医学影像,少样本也能超顶尖模型。

Tri-Plane Mamba: Efficiently Adapting Segment Anything Model for 3D Medical Images

  • 用多尺度3D卷积和三平面Mamba模块处理深度信息,兼顾效率与长程依赖。
  • 仅用3个样本训练,3DCT器官分割Dice提升达12%以上,超越传统模型。
  • 适合数据稀缺的医疗图像分割场景,尤其适合资源有限的研究者。

3D医学图像分割的通用网络虽表现优异,但依赖大量像素级标注数据,耗时费力。尽管分割一切模型(SAM)在2D医学图像上通过参数与数据高效适配取得卓越效果,但3D图像引入额外深度通道,既无法复用2D预训练特征,又使适配计算成本呈平方增长。为此,本文提出专为SAM设计的三平面Mamba(TP-Mamba)适配器,包含两大创新:1)多尺度3D卷积适配器,高效处理局部深度信息;2)三平面Mamba模块,低成本捕捉长程深度表征。该方法在BTCV数据集的3D CT器官分割任务中达到当前最优性能,尤其在极低数据条件下表现突出——仅使用3个训练样本,其Dice分数相比传统3D分割网络最高提升12%。

原文摘要 · Abstract (English)

General networks for 3D medical image segmentation have recently undergone extensive exploration. Behind the exceptional performance of these networks lies a significant demand for a large volume of pixel-level annotated data, which is time-consuming and labor-intensive. The emergence of the Segment Anything Model (SAM) has enabled this model to achieve superior performance in 2D medical image segmentation tasks via parameter- and data-efficient feature adaptation. However, the introduction of additional depth channels in 3D medical images not only prevents the sharing of 2D pre-trained features but also results in a quadratic increase in the computational cost for adapting SAM. To overcome these challenges, we present the Tri-Plane Mamba (TP-Mamba) adapters tailored for the SAM, featuring two major innovations: 1) multi-scale 3D convolutional adapters, optimized for efficiently processing local depth-level information, 2) a tri-plane mamba module, engineered to capture long-range depth-level representation without significantly increasing computational costs. This approach achieves state-of-the-art performance in 3D CT organ segmentation tasks. Remarkably, this superior performance is maintained even with scarce training data. Specifically using only three CT training samples from the BTCV dataset, it surpasses conventional 3D segmentation networks, attaining a Dice score that is up to 12% higher.

3D分割SAM轻量化医学影像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。