统一处理2D/3D医学图像多模态分割，无需微调即可跨任务泛化
UniMedSeg: Unified In-Context Learning for Multi-Paradigm 2D/3D Medical Image Segmentation
- 将视觉、交互、语言等不同分割方式映射到统一序列空间
- 在27个数据集上实现跨模态、跨维度的最优性能
- 适合需要通用医学图像分割能力的研究与临床应用
医学图像分割基础模型需在多样临床场景中具备泛化能力，但现有通用方法仍受提示范式和空间维度限制。视觉上下文学习、交互式分割与语言引导分割通常由专用模型处理，2D与3D图像也分别建模。这种隔离导致异构标注与数据难以被单一可扩展模型联合吸收，且阻碍跨范式知识迁移。为此，我们提出UniMedSeg，一种以Transformer为核心的通用分割框架，将视觉样本、几何交互、语言指令及2D/3D图像统一映射至共享序列空间，通过统一上下文接口联合学习异构医学监督信号，无需针对任务或维度设计分支。为克服视觉上下文引发的长序列记忆瓶颈，引入解耦分块注意力机制，将注意力复杂度降至线性，同时保持硬件友好计算与聚焦的上下文-目标交互。在涵盖27个公开数据集的大规模语料上训练与评估，UniMedSeg在视觉上下文、交互式与语言引导分割任务中均达到当前最优表现，且无需任务特定微调，展现出强泛化能力。代码与模型权重已开源。
原文摘要 · Abstract (English)
Medical image segmentation foundation models are expected to generalize across diverse clinical scenarios, yet existing universal methods remain fragmented by prompt paradigms and spatial dimensions. Visual in-context learning, interactive segmentation, and language-guided segmentation are typically handled by paradigm-specific models, while 2D and 3D images are also modeled separately. Such isolation prevents heterogeneous annotations and data from being jointly absorbed by a single scalable model and limits cross-paradigm knowledge transfer. To address this bottleneck, we propose UniMedSeg, a Transformer-centric universal segmentation framework that maps visual examples, geometric interactions, language instructions, and 2D/3D images into a shared sequence space, enabling heterogeneous medical supervision to be jointly learned through a unified in-context interface without prompt- or dimension-specific branches. To overcome the long-sequence memory bottleneck caused by visual contexts, we introduce Decoupled Split Attention, which reduces attention complexity to linear while preserving hardware-friendly computation and focused context-target interaction. Extensively trained and evaluated on a large corpus curated from 27 public datasets, UniMedSeg achieves state-of-the-art performance across visual in-context, interactive, and language-guided segmentation without task-specific fine-tuning, demonstrating strong generalization on diverse held-out tasks. The code and model weights are publicly available at https://github.com/Lii1228/UniMedSeg
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。