arXiv:2510.19239eess.IV2025-10被引 4

轻量级超声基础模型,高效部署且性能不降。

TinyUSFM: Towards Compact and Efficient Ultrasound Foundation Models

  • 用精选小数据集+知识蒸馏,压缩大模型。
  • 仅用20万张图,参数减至6.36%,性能持平。
  • 适合资源受限的临床环境快速部署。

医学影像基础模型在多种解剖结构和临床任务中表现出卓越泛化能力,但其高性能依赖大量计算资源,限制了在资源有限的临床场景中的应用。本文提出TinyUSFM,首个轻量级超声基础模型,通过知识蒸馏结合精心筛选的小规模数据集,在保持大型超声基础模型(USFM)的器官泛化性和任务适应性的同时,显著提升计算效率。针对轻量模型容量与表征能力有限的问题,提出基于特征梯度的协同数据集选择策略,选取高质量紧凑训练数据,避免低质量冗余图像导致的性能下降。为保留知识迁移中的空间与频域特性,设计领域分离的掩码图像建模辅助的一致性驱动动态蒸馏框架,通过教师模型在不同领域掩码下的一致性自适应地传递知识,专用于超声图像解读。评估方面,构建了目前最大的公开超声基准测试集UniUS-Bench,涵盖15个器官的8个分类与10个分割任务。仅使用20万张图像进行蒸馏,TinyUSFM在参数量仅为USFM的6.36%、计算量仅6.40%的情况下,达到相当性能;分类任务比原始轻量模型高9.45%,分割任务高7.72%,超越所有现有轻量级模型,在跨设备与中心的平均分类准确率达84.91%,平均分割Dice得分为85.78%。

原文摘要 · Abstract (English)

Foundation models for medical imaging demonstrate superior generalization capabilities across diverse anatomical structures and clinical applications. Their outstanding performance relies on substantial computational resources, limiting deployment in resource-constrained clinical environments. This paper presents TinyUSFM, the first lightweight ultrasound foundation model that maintains superior organ versatility and task adaptability of our large-scale Ultrasound Foundation Model (USFM) through knowledge distillation with strategically curated small datasets, delivering significant computational efficiency without sacrificing performance. Considering the limited capacity and representation ability of lightweight models, we propose a feature-gradient driven coreset selection strategy to curate high-quality compact training data, avoiding training degradation from low-quality redundant images. To preserve the essential spatial and frequency domain characteristics during knowledge transfer, we develop domain-separated masked image modeling assisted consistency-driven dynamic distillation. This novel framework adaptively transfers knowledge from large foundation models by leveraging teacher model consistency across different domain masks, specifically tailored for ultrasound interpretation. For evaluation, we establish the UniUS-Bench, the largest publicly available ultrasound benchmark comprising 8 classification and 10 segmentation datasets across 15 organs. Using only 200K images in distillation, TinyUSFM matches USFM's performance with just 6.36% of parameters and 6.40% of GFLOPs. TinyUSFM significantly outperforms the vanilla model by 9.45% in classification and 7.72% in segmentation, surpassing all state-of-the-art lightweight models, and achieving 84.91% average classification accuracy and 85.78% average segmentation Dice score across diverse medical devices and centers.

超声成像轻量模型知识蒸馏医学AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。