Medal S 用空间+文本联合提示，实现高精度多模态医学图像分割。
Medal S: Spatio-Textual Prompt Model for Medical Segmentation
- 融合3D空间与文本提示，通道对齐避免分辨率失真。
- 支持243类分割，五模态平均Dice达75.44，提速超90%。
- 自优化文本模式适合无标注场景，适合临床高效部署。
我们提出Medal S，一种支持原生分辨率空间与文本提示的医学分割基础模型，可在端到端可训练框架中实现。不同于仅依赖文本的方法，Medal S通过体素级通道对齐实现空间与文本嵌入的精准匹配，缓解分辨率不一致导致的误差。通过保留完整3D上下文，能并行处理多个原生分辨率掩码，显著提升多类别分割性能。轻量级3D卷积模块在体素空间进行精细优化，结合两种提示类型，支持在BiomedSegFM数据集上覆盖CT、MRI、PET、超声和显微镜共243类。模型提供两种提示模式：纯文本模式下，模型预测作为空间提示实现无需人工输入的自优化；混合模式则结合人工标注以增强灵活性。针对24类分割任务，并行空间提示使推理时间减少超过90%。我们提出动态重采样以解决目标块比例失衡问题，扩展SAT与nnU-Net用于数据增强。此外，优化了文本预处理、两阶段推理策略及后处理技术，提升内存效率、精度与推理速度。在五模态验证集平均表现上，Medal S超越SAT：Dice为75.44（对比69.83），NSD为77.34（对比71.06），F1为38.24（对比24.88），Dice TP为65.46（对比46.97）。Medal S通过空间精度与语义文本引导的协同，展现出优于传统序列提示方法的卓越效率与准确性。代码将公开于https://github.com/yinghemedical/Medal-S。
原文摘要 · Abstract (English)
We introduce Medal S, a medical segmentation foundation model that supports native-resolution spatial and textual prompts within an end-to-end trainable framework. Unlike text-only methods lacking spatial awareness, Medal S achieves channel-wise alignment between volumetric prompts and text embeddings, mitigating inaccuracies from resolution mismatches. By preserving full 3D context, it efficiently processes multiple native-resolution masks in parallel, enhancing multi-class segmentation performance. A lightweight 3D convolutional module enables precise voxel-space refinement guided by both prompt types, supporting up to 243 classes across CT, MRI, PET, ultrasound, and microscopy modalities in the BiomedSegFM dataset. Medal S offers two prompting modes: a text-only mode, where model predictions serve as spatial prompts for self-refinement without human input, and a hybrid mode, incorporating manual annotations for enhanced flexibility. For 24-class segmentation, parallel spatial prompting reduces inference time by more than 90% compared to sequential prompting. We propose dynamic resampling to address target-patch ratio imbalance, extending SAT and nnU-Net for data augmentation. Furthermore, we develop optimized text preprocessing, a two-stage inference strategy, and post-processing techniques to improve memory efficiency, precision, and inference speed. On the five-modality average on the validation set, Medal S outperforms SAT with a DSC of 75.44 (vs. 69.83), NSD of 77.34 (vs. 71.06), F1 of 38.24 (vs. 24.88), and DSC TP of 65.46 (vs. 46.97). Medal S achieves excellent performance by harmonizing spatial precision with semantic textual guidance, demonstrating superior efficiency and accuracy in multi-class medical segmentation tasks compared to sequential prompt-based approaches. Medal S will be publicly available at https://github.com/yinghemedical/Medal-S.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。