无需微调和专家标注,用少量样本即可让SAM2实现3D医学图像精准分割
Few-Shot Adaptation of Training-Free Foundation Model for 3D Medical Image Segmentation
- 重用SAM2预训练模块,仅靠少量样本就可捕捉解剖知识
- 在多个数据集上达到接近微调模型的精度,且无需人工提示
- 适合临床场景快速部署,尤其适用于标注数据稀缺的医学图像任务
视觉基础模型在图像分析任务中取得了显著进展。以分割一切模型(SAM)为代表的模型可通过用户提供的提示实现通用零样本分割。然而,SAM主要基于自然图像训练,缺乏医学影像领域的专有知识。这导致其在医学图像分割中面临两大挑战:需在专业医学数据集上进行大量微调,以及依赖人工提示,耗时且需医学专家参与。本文提出一种无训练的少样本适配方法FATE-SAM,用于3D医学图像分割。FATE-SAM通过重构SAM2的预训练模块,仅用少量支持样本即可捕获解剖结构知识,并实现免提示分割,无需模型微调。为应对医学图像的体数据特性,引入体素一致性机制以增强三维切片间的空间连贯性。在多个医学影像数据集上的评估表明,FATE-SAM在无需大规模标注数据和专家干预的情况下,仍能实现鲁棒且准确的分割性能,优于监督学习、零样本SAM及微调后的医学专用SAM方法。该方法为临床应用提供了高效实用的解决方案。
原文摘要 · Abstract (English)
Vision foundation models have achieved remarkable progress across various image analysis tasks. In the image segmentation task, foundation models like the Segment Anything Model (SAM) enable generalizable zero-shot segmentation through user-provided prompts. However, SAM primarily trained on natural images, lacks the domain-specific expertise of medical imaging. This limitation poses challenges when applying SAM to medical image segmentation, including the need for extensive fine-tuning on specialized medical datasets and a dependency on manual prompts, which are both labor-intensive and require intervention from medical experts. This work introduces the Few-shot Adaptation of Training-frEe SAM (FATE-SAM), a novel method designed to adapt the advanced Segment Anything Model 2 (SAM2) for 3D medical image segmentation. FATE-SAM reassembles pre-trained modules of SAM2 to enable few-shot adaptation, leveraging a small number of support examples to capture anatomical knowledge and perform prompt-free segmentation, without requiring model fine-tuning. To handle the volumetric nature of medical images, we incorporate a Volumetric Consistency mechanism that enhances spatial coherence across 3D slices. We evaluate FATE-SAM on multiple medical imaging datasets and compare it with supervised learning methods, zero-shot SAM approaches, and fine-tuned medical SAM methods. Results show that FATE-SAM delivers robust and accurate segmentation while eliminating the need for large annotated datasets and expert intervention. FATE-SAM provides a practical, efficient solution for medical image segmentation, making it more accessible for clinical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。