用文本描述精准分割医学影像，免去繁琐标注。
MedSAM3: Delving into Segment Anything with Medical Concepts
- 基于SAM3架构，用语义标签微调实现开集文本提示分割。
- 跨模态医学影像分割效果超越现有专用与基础模型。
- 适合需要快速部署、无需大量标注的医疗AI研发人员。
医学图像分割是生物医学发现的基础。现有方法泛化能力差，且新临床应用需耗时耗力的手动标注。本文提出MedSAM-3，一种可接受文本提示的医学图像与视频分割模型。通过在配对了语义概念标签的医学图像上微调Segment Anything Model（SAM）3架构，我们的MedSAM-3实现了医学提示性概念分割（PCS），可通过开放词汇的文本描述精确定位解剖结构，而不仅依赖几何提示。我们进一步引入MedSAM-3 Agent框架，集成多模态大语言模型（MLLMs），在“人机协同”工作流中实现复杂推理与迭代优化。在涵盖X光、MRI、超声、CT及视频等多种医学影像模态的全面实验中，本方法显著优于现有专业模型与基础模型。代码与模型将开源至https://github.com/Joey-S-Liu/MedSAM3。
原文摘要 · Abstract (English)
Medical image segmentation is fundamental for biomedical discovery. Existing methods lack generalizability and demand extensive, time-consuming manual annotation for new clinical application. Here, we propose MedSAM-3, a text promptable medical segmentation model for medical image and video segmentation. By fine-tuning the Segment Anything Model (SAM) 3 architecture on medical images paired with semantic conceptual labels, our MedSAM-3 enables medical Promptable Concept Segmentation (PCS), allowing precise targeting of anatomical structures via open-vocabulary text descriptions rather than solely geometric prompts. We further introduce the MedSAM-3 Agent, a framework that integrates Multimodal Large Language Models (MLLMs) to perform complex reasoning and iterative refinement in an agent-in-the-loop workflow. Comprehensive experiments across diverse medical imaging modalities, including X-ray, MRI, Ultrasound, CT, and video, demonstrate that our approach significantly outperforms existing specialist and foundation models. We will release our code and model at https://github.com/Joey-S-Liu/MedSAM3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。