用弱监督自动学习提示，让医学图像分割无需人工标注框
Automating MedSAM by Learning Prompts with Weak Few-Shot Supervision
- 用轻量模块从图像直接生成提示嵌入，替代手动输入框
- 仅需10个样本+紧致边界框，即可实现自动分割
- 适用于医疗影像，降低标注成本，适合快速部署
如最近提出的通用图像分割模型SAM，在图像分割任务中表现卓越。但其通常需要用户通过手工设计的提示（如边界框）进行交互，限制了在下游任务中的应用。若使用全标注数据微调，则需大量人工标注，成本高昂。本文提出用轻量级模块直接从图像嵌入中学习提示嵌入，替代输入提示。该嵌入与图像嵌入共同驱动基础模型输出分割掩码。所提方法在针对医学图像优化的MedSAM上验证，涵盖三种磁共振和超声影像数据集。仅需10个样本及弱标签（紧致边界框），即可实现自动分割。代码已开源：https://github.com/Minimel/MedSAMWeakFewShotPromptAutomation。
原文摘要 · Abstract (English)
Foundation models such as the recently introduced Segment Anything Model (SAM) have achieved remarkable results in image segmentation tasks. However, these models typically require user interaction through handcrafted prompts such as bounding boxes, which limits their deployment to downstream tasks. Adapting these models to a specific task with fully labeled data also demands expensive prior user interaction to obtain ground-truth annotations. This work proposes to replace conditioning on input prompts with a lightweight module that directly learns a prompt embedding from the image embedding, both of which are subsequently used by the foundation model to output a segmentation mask. Our foundation models with learnable prompts can automatically segment any specific region by 1) modifying the input through a prompt embedding predicted by a simple module, and 2) using weak labels (tight bounding boxes) and few-shot supervision (10 samples). Our approach is validated on MedSAM, a version of SAM fine-tuned for medical images, with results on three medical datasets in MR and ultrasound imaging. Our code is available on https://github.com/Minimel/MedSAMWeakFewShotPromptAutomation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。