构建最大公开超声分割数据集,训练出高效超声基础模型UltraSam。
UltraSam: A Foundation Model for Ultrasound using Large Open-Access Segmentation Datasets
- 基于43个公开数据集构建28万+图像的超声分割数据集US-43d。
- UltraSam在多个数据集上显著优于现有SAM类模型,支持点/框提示。
- 可作为下游任务初始化模型,适用于超声影像分割与分类任务。
自动化超声图像分析因解剖结构复杂且标注数据有限而面临挑战。为此,我们采用数据驱动方法,整合迄今最大的公开超声分割数据集,并训练一个专为超声设计的通用视觉基础模型。我们构建了US-43d,涵盖43个公开超声数据集,包含超过280,000张图像及50多个解剖结构的分割掩码。在此基础上,我们提出UltraSam,即基于Segment Anything Model(SAM)的改进模型,可在US-43d上训练,并支持点提示与框提示。此外,我们提出一种新应用场景:将UltraSam作为模型初始化,用于微调多种下游分析任务,验证其作为基础模型的能力。结果表明,UltraSam在三个不同公共数据集上的提示式分割任务中性能远超现有SAM类模型;使用UltraSam初始化的视觉变换器,在多种下游分割与分类任务中表现优于ImageNet、SAM和MedSAM初始化模型,充分体现了其作为基础模型的有效性。我们发布了代码与预训练模型(https://github.com/CAMMA-public/UltraSam),并邀请社区贡献高质量数据集以推动该领域发展。
原文摘要 · Abstract (English)
Purpose: Automated ultrasound image analysis is challenging due to anatomical complexity and limited annotated data. To tackle this, we take a data-centric approach, assembling the largest public ultrasound segmentation dataset and training a versatile visual foundation model tailored for ultrasound. Methods: We compile US-43d, a large-scale collection of 43 open-access ultrasound datasets with over 280,000 images and segmentation masks for more than 50 anatomical structures. We then introduce UltraSam, an adaptation of the Segment Anything Model (SAM) that is trained on US-43d and supports both point- and box-prompts. Finally, we introduce a new use case for SAM-style models by using UltraSam as a model initialization that can be fine-tuned for various downstream analysis tasks, demonstrating UltraSam's foundational capabilities. Results: UltraSam achieves vastly improved performance over existing SAM-style models for prompt-based segmentation on three diverse public datasets. Moreover, an UltraSam-initialized Vision Transformer surpasses ImageNet-, SAM-, and MedSAM-initialized models in various downstream segmentation and classification tasks, highlighting UltraSam's effectiveness as a foundation model. Conclusion: We compile US-43d, a large-scale unified ultrasound dataset, and introduce UltraSam, a powerful multi-purpose SAM-style model for ultrasound images. We release our code and pretrained models at https://github.com/CAMMA-public/UltraSam and invite the community to further this effort by contributing high-quality datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。