用部分标注数据实现多模态医学图像分割,标注成本减半。
Partially Supervised Unpaired Multi-Modal Learning for Label-Efficient Medical Image Segmentation
- 设计新框架DEST,分解处理部分标签带来的类别分布差异。
- 在心脏和腹部器官分割任务中,性能显著优于现有方法。
- 适合标注资源有限的医疗影像研究者使用。
无配对多模态学习(UMML)通过利用未配对的多模态数据提升单模态模型性能,受到医学图像分析领域的广泛关注。然而,现有方法要求数据集完全标注,带来巨大标注成本。本文研究部分标注数据下的标签高效无配对多模态学习,可将标注成本降低一半。提出新的学习范式——部分监督无配对多模态学习(PSUMML),并构建分解式部分类别自适应与快照集成自训练(DEST)框架。该框架采用带有模态特异性归一化层的紧凑分割网络,以支持部分标注的无配对多模态学习。核心挑战在于部分标签导致的复杂类别分布偏差,阻碍跨模态知识迁移。我们通过分解定理进行理论分析,并提出分解式部分类别自适应技术,精确对齐不同模态间的部分标注类别,减少分布偏差。进一步提出快照集成自训练技术,利用训练过程中的快照模型为部分标注像素生成伪标签,实现自训练以提升模型性能。我们在心脏亚结构分割与腹部多器官分割两个任务上,在多种PSUMML场景下进行大量实验,结果表明该框架显著优于现有方法。
原文摘要 · Abstract (English)
Unpaired Multi-Modal Learning (UMML) which leverages unpaired multi-modal data to boost model performance on each individual modality has attracted a lot of research interests in medical image analysis. However, existing UMML methods require multi-modal datasets to be fully labeled, which incurs tremendous annotation cost. In this paper, we investigate the use of partially labeled data for label-efficient unpaired multi-modal learning, which can reduce the annotation cost by up to one half. We term the new learning paradigm as Partially Supervised Unpaired Multi-Modal Learning (PSUMML) and propose a novel Decomposed partial class adaptation with snapshot Ensembled Self-Training (DEST) framework for it. Specifically, our framework consists of a compact segmentation network with modality specific normalization layers for learning with partially labeled unpaired multi-modal data. The key challenge in PSUMML lies in the complex partial class distribution discrepancy due to partial class annotation, which hinders effective knowledge transfer across modalities. We theoretically analyze this phenomenon with a decomposition theorem and propose a decomposed partial class adaptation technique to precisely align the partially labeled classes across modalities to reduce the distribution discrepancy. We further propose a snapshot ensembled self-training technique to leverage the valuable snapshot models during training to assign pseudo-labels to partially labeled pixels for self-training to boost model performance. We perform extensive experiments under different scenarios of PSUMML for two medical image segmentation tasks, namely cardiac substructure segmentation and abdominal multi-organ segmentation. Our framework outperforms existing methods significantly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。