用文本提示实现超声多器官分割，无需额外标注即可泛化到新数据。
Grounding DINO-US-SAM: Text-Prompted Multi-Organ Segmentation in Ultrasound with LoRA-Tuned Vision-Language Models
- 结合视觉语言模型与SAM2，通过文本提示驱动分割。
- 在18个公开数据集上表现超越现有方法，未见数据集也保持稳定性能。
- 仅需少量微调，适合临床快速部署的医学影像分析场景。
超声图像中准确且泛化的目标分割仍面临挑战，主要源于解剖结构差异、成像协议多样及标注数据有限。本文提出一种基于文本提示的视觉-语言模型（VLM），将Grounding DINO与SAM2融合，实现多器官超声图像分割。共使用18个公开超声数据集，涵盖乳腺、甲状腺、肝脏、前列腺、肾脏和腰肌等器官。其中15个用于基于低秩适应（LoRA）的Grounding DINO微调与验证，3个完全保留用于测试以评估对未见分布的表现。实验表明，该方法在多数已见数据集上优于UniverSeg、MedSAM、MedCLIP-SAM、BiomedParse和SAMUS等先进方法，并在未见数据集上无需额外微调仍保持良好性能。结果表明，视觉-语言模型在可扩展、鲁棒的超声图像分析中具有巨大潜力，显著降低对大规模特定器官标注数据的依赖。代码将于论文接收后发布于code.sonography.ai。
原文摘要 · Abstract (English)
Accurate and generalizable object segmentation in ultrasound imaging remains a significant challenge due to anatomical variability, diverse imaging protocols, and limited annotated data. In this study, we propose a prompt-driven vision-language model (VLM) that integrates Grounding DINO with SAM2 (Segment Anything Model2) to enable object segmentation across multiple ultrasound organs. A total of 18 public ultrasound datasets, encompassing the breast, thyroid, liver, prostate, kidney, and paraspinal muscle, were utilized. These datasets were divided into 15 for fine-tuning and validation of Grounding DINO using Low Rank Adaptation (LoRA) to the ultrasound domain, and 3 were held out entirely for testing to evaluate performance in unseen distributions. Comprehensive experiments demonstrate that our approach outperforms state-of-the-art segmentation methods, including UniverSeg, MedSAM, MedCLIP-SAM, BiomedParse, and SAMUS on most seen datasets while maintaining strong performance on unseen datasets without additional fine-tuning. These results underscore the promise of VLMs in scalable and robust ultrasound image analysis, reducing dependence on large, organ-specific annotated datasets. We will publish our code on code.sonography.ai after acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。