用语音转文本指导医学影像分割,参数少、效果接近全量微调。
LoGSAM: Parameter-Efficient Cross-Modal Grounding for MRI Segmentation
- 通过语音转文本生成提示词,驱动视觉语言模型定位病灶。
- 在BRISC 2025上达80.32%的Dice分数,仅用5%参数量。
- 支持医生语音输入,适合临床快速部署的精准分割场景。
利用磁共振成像(MRI)精确识别和勾画脑肿瘤对治疗规划和手术决策至关重要。为此,我们提出LoGSAM——一种参数高效的检测驱动框架,将放射科医生口述转化为文本提示,用于基于基础模型的定位与分割。先用预训练Whisper ASR模型转录并翻译医生语音,再通过具备否定识别能力的临床NLP提取肿瘤特异性文本提示。这些提示引导经LoRA适配的视觉-语言检测模型Grounding DINO(GDINO)完成肿瘤定位,生成的边界框作为提示输入MedSAM,直接生成像素级肿瘤掩码,无需额外微调。在BRISC 2025数据集上,LoGSAM达到80.32%的Dice分数,达到全量微调的GDINO+MedSAM基线的98.6%,而训练参数不足其5%。此外,在未见过的德语口述数据上,病例级分类提取准确率达91.7%。结果表明,仅通过少量参数更新即可构建模块化语音到分割流水线。
原文摘要 · Abstract (English)
Precise localization and delineation of brain tumors using magnetic resonance imaging (MRI) are essential for planning therapy and guiding surgical decisions. To address this, we propose LoGSAM, a parameter-efficient, detection-driven framework that transforms radiologist dictation into text prompts for foundation-model-based localization and segmentation. Radiologist speech is first transcribed and translated using a pretrained Whisper ASR model, followed by negation-aware clinical NLP to extract tumor-specific textual prompts. These prompts guide text-conditioned tumor localization via a LoRA-adapted vision-language detection model, Grounding DINO (GDINO). The predicted bounding boxes are used as prompts for MedSAM to generate pixel-level tumor masks without any additional fine-tuning. On BRISC 2025, LoGSAM attains a Dice score of 80.32%, reaching 98.6% of a fully fine-tuned GDINO + MedSAM baseline while training fewer than 5\% of its parameters, indicating a favorable accuracy/parameter trade-off. In addition, we evaluate the full pipeline using German dictations from a board-certified radiologist on unseen MRI scans, achieving 91.7\% case-level class-extraction accuracy. These results highlight the feasibility of constructing a modular speech-to-segmentation pipeline from pretrained foundation models with minimal parameter updates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。