用视觉大模型实现高精度实时超声图像分割，少样本下表现更优。
Adapting Vision Foundation Models for Real-time Ultrasound Image Segmentation
- 融合Hiera与DINOv2提取多尺度特征，增强图像表达能力。
- 1%和10%数据下比nnUNet平均提升超20%，6个公开数据集表现领先。
- 单卡推理速度达77帧/秒，适合临床实时应用。
本文提出一种新方法，将分层视觉基础模型Hiera适配用于实时超声图像分割。现有超声分割方法在新任务上适应性差，依赖昂贵的人工标注；而实时方法通常难以达到顶尖性能。为此，我们设计了一个自适应框架，利用Hiera提取多尺度特征，并融合DINOv2表示以增强视觉表征力。经解码后生成精确且鲁棒的分割结果。我们在六个公开数据集和一个内部数据集上进行了广泛评估，涵盖心脏与甲状腺超声分割任务。实验表明，该方法在多个数据集上均超越现有最优方法，在1%和10%数据设置下平均超过nnUNet 20%以上。采用TensorRT在单张GPU上实现约77 FPS的推理速度，满足临床实时需求。
原文摘要 · Abstract (English)
We propose a novel approach that adapts hierarchical vision foundation models for real-time ultrasound image segmentation. Existing ultrasound segmentation methods often struggle with adaptability to new tasks, relying on costly manual annotations, while real-time approaches generally fail to match state-of-the-art performance. To overcome these limitations, we introduce an adaptive framework that leverages the vision foundation model Hiera to extract multi-scale features, interleaved with DINOv2 representations to enhance visual expressiveness. These enriched features are then decoded to produce precise and robust segmentation. We conduct extensive evaluations on six public datasets and one in-house dataset, covering both cardiac and thyroid ultrasound segmentation. Experiments show that our approach outperforms state-of-the-art methods across multiple datasets and excels with limited supervision, surpassing nnUNet by over 20\% on average in the 1\% and 10\% data settings. Our method achieves $\sim$77 FPS inference speed with TensorRT on a single GPU, enabling real-time clinical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。