arXiv:2502.19671cs.CV2025-02

提出测试时跨模态泛化框架,提升医学图像分割在未见模态下的性能。

Test-Time Modality Generalization for Medical Image Segmentation

  • 引入模态感知风格投影与敏感实例归一化,增强对未知模态的适应能力。
  • 在11个数据集上跨4种模态(内镜、超声、皮肤镜、影像)均优于现有方法。
  • 适合临床部署中需应对多种设备或成像方式的医学图像分割任务。

可泛化的医学图像分割对于保证在不同未见临床场景下性能一致至关重要。然而,现有方法往往忽视了在任意未见模态上的泛化能力。本文提出一种新的测试时模态泛化(TTMG)框架,包含两个核心组件:模态感知风格投影(MASP)和模态敏感实例归一化(MSIW),旨在提升对任意未见模态数据集的泛化能力。MASP通过估计测试样本属于各已见模态的可能性,并利用模态特定风格基底将其映射到对应分布,实现有效投影。此外,由于高特征协方差会阻碍对未见模态的泛化,因此在训练中引入MSIW,选择性抑制模态敏感信息,同时保留模态不变特征。结合两者,TTMG框架在未见模态的医学图像分割任务中表现出稳健的泛化性能,这一挑战目前多数方法尚未充分解决。我们在涵盖四种模态(内镜、超声、皮肤镜、放射学)的11个数据集上评估了TTMG,与多种域泛化技术相比,在多种模态组合下持续取得更优的分割表现。

原文摘要 · Abstract (English)

Generalizable medical image segmentation is essential for ensuring consistent performance across diverse unseen clinical settings. However, existing methods often overlook the capability to generalize effectively across arbitrary unseen modalities. In this paper, we introduce a novel Test-Time Modality Generalization (TTMG) framework, which comprises two core components: Modality-Aware Style Projection (MASP) and Modality-Sensitive Instance Whitening (MSIW), designed to enhance generalization in arbitrary unseen modality datasets. The MASP estimates the likelihood of a test instance belonging to each seen modality and maps it onto a distribution using modality-specific style bases, guiding its projection effectively. Furthermore, as high feature covariance hinders generalization to unseen modalities, the MSIW is applied during training to selectively suppress modality-sensitive information while retaining modality-invariant features. By integrating MASP and MSIW, the TTMG framework demonstrates robust generalization capabilities for medical image segmentation in unseen modalities a challenge that current methods have largely neglected. We evaluated TTMG alongside other domain generalization techniques across eleven datasets spanning four modalities (colonoscopy, ultrasound, dermoscopy, and radiology), consistently achieving superior segmentation performance across various modality combinations.

医学图像分割泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。