用SAM3实现跨模态病灶分割，支持概念驱动精准定位。
Adapting Segment Anything Model 3 for Concept-Driven Lesion Segmentation in Medical Images: An Experimental Study
- 基于文本和图像提示的concept-driven分割方法
- 在13个数据集上实现跨模态高精度病灶勾画
- 适合需要快速适配新病灶类型的临床研究者
准确的病灶分割对医学图像分析至关重要，但现有方法多针对特定解剖部位或成像模态，泛化能力有限。最近的视觉-语言基础模型可在自然图像中实现概念驱动分割，为更灵活的医学图像分析提供了新方向。然而，基于概念提示的病灶分割，尤其是最新的Segment Anything Model 3（SAM3），仍缺乏系统探索。本文对SAM3在病灶分割中的表现进行了系统评估，使用几何边界框及基于概念的文本和图像提示，在多模态数据（包括多参数MRI、CT、超声、皮肤镜、内窥镜）上测试其性能。为提升鲁棒性，引入邻近切片预测、多参数信息及先验标注等额外先验知识。进一步比较了部分模块微调、基于适配器的方法与全模型优化等不同微调策略。在覆盖11类病灶的13个数据集上的实验表明，SAM3展现出强大的跨模态泛化能力、可靠的基于概念的分割性能以及精确的病灶边界描绘。结果凸显了基于概念的基础模型在可扩展、实用化医学图像分割中的潜力。代码与训练模型将发布于：https://github.com/apple1986/lesion-sam3
原文摘要 · Abstract (English)
Accurate lesion segmentation is essential in medical image analysis, yet most existing methods are designed for specific anatomical sites or imaging modalities, limiting their generalizability. Recent vision-language foundation models enable concept-driven segmentation in natural images, offering a promising direction for more flexible medical image analysis. However, concept-prompt-based lesion segmentation, particularly with the latest Segment Anything Model 3 (SAM3), remains underexplored. In this work, we present a systematic evaluation of SAM3 for lesion segmentation. We assess its performance using geometric bounding boxes and concept-based text and image prompts across multiple modalities, including multiparametric MRI, CT, ultrasound, dermoscopy, and endoscopy. To improve robustness, we incorporate additional prior knowledge, such as adjacent-slice predictions, multiparametric information, and prior annotations. We further compare different fine-tuning strategies, including partial module tuning, adapter-based methods, and full-model optimization. Experiments on 13 datasets covering 11 lesion types demonstrate that SAM3 achieves strong cross-modality generalization, reliable concept-driven segmentation, and accurate lesion delineation. These results highlight the potential of concept-based foundation models for scalable and practical medical image segmentation. Code and trained models will be released at: https://github.com/apple1986/lesion-sam3
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。