arXiv:2503.19359cs.CV2025-03CVPR被引 21

无需微调,用参考图即可通用分割医学图像

Show and Segment: Universal Medical Image Segmentation via In-Context Learning

  • 通过参考图像对提取任务信息,动态引导分割
  • 在12个数据集上表现接近专用模型,在7个新数据集上泛化更强
  • 能自动发现解剖结构关系,适合跨模态研究者使用

医学图像分割因解剖结构、成像模态和任务多样性而极具挑战。尽管深度学习已取得进展,现有方法仍需针对特定任务训练或微调,难以泛化。我们提出Iris——一种基于上下文参考图像的分割框架,可在不微调的情况下灵活适应新任务。其核心为轻量级上下文任务编码模块,从参考图像-标签对中提炼任务特异性信息,并用于指导目标对象分割。通过解耦任务编码与推理,Iris支持单样本推理、参考例集合、对象级参考检索及上下文微调等多种策略。在12个数据集上的综合评估显示,Iris在分布内任务上性能接近专用模型;在7个未见数据集上,对分布外数据和未见类别展现出更优泛化能力。此外,其任务编码模块可自动发现跨数据集与模态的解剖关系,为医学对象提供无需显式解剖监督的洞察。

原文摘要 · Abstract (English)

Medical image segmentation remains challenging due to the vast diversity of anatomical structures, imaging modalities, and segmentation tasks. While deep learning has made significant advances, current approaches struggle to generalize as they require task-specific training or fine-tuning on unseen classes. We present Iris, a novel In-context Reference Image guided Segmentation framework that enables flexible adaptation to novel tasks through the use of reference examples without fine-tuning. At its core, Iris features a lightweight context task encoding module that distills task-specific information from reference context image-label pairs. This rich context embedding information is used to guide the segmentation of target objects. By decoupling task encoding from inference, Iris supports diverse strategies from one-shot inference and context example ensemble to object-level context example retrieval and in-context tuning. Through comprehensive evaluation across twelve datasets, we demonstrate that Iris performs strongly compared to task-specific models on in-distribution tasks. On seven held-out datasets, Iris shows superior generalization to out-of-distribution data and unseen classes. Further, Iris's task encoding module can automatically discover anatomical relationships across datasets and modalities, offering insights into medical objects without explicit anatomical supervision.

医学图像零样本上下文学习分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。