解决脑肿瘤分割中模态缺失问题，提升不完整MRI数据的分割精度。
Uni-Encoder Meets Multi-Encoders: Representation Before Fusion for Brain Tumor Segmentation with Missing Modalities
- 先用统一视觉编码器预训练，再加特定模态的多编码器提取细节特征。
- 在BraTS 2023和2024上均优于现有方法，尤其在模态缺失时表现更稳。
- 适合临床实际中模态不全的脑肿瘤分割场景，可推广至其他医学影像任务。
多模态MRI能提供互补信息用于脑肿瘤分割，但临床扫描常缺失一个或多个模态，导致分割性能下降。本文提出UniME（Uni-Encoder Meets Multi-Encoders），一种两阶段异构方法，用于处理模态缺失下的脑肿瘤分割，平衡细粒度结构捕捉、跨模态互补建模与可用模态利用之间的权衡。核心思路是将表示学习与分割解耦：第一阶段使用掩码图像建模预训练单一ViT编码器（Uni-Encoder），构建对模态缺失鲁棒的统一表征；第二阶段引入特定模态的CNN多编码器（Multi-Encoders），提取高分辨率、多尺度的细粒度特征。最终融合全局表征与局部特征，生成精准分割结果。在BraTS 2023和BraTS 2024上的实验表明，该方法在不完备多模态条件下优于现有方法。代码已公开于https://github.com/Hooorace-S/UniME。
原文摘要 · Abstract (English)
Multimodal MRI offers complementary information for brain tumor segmentation, but clinical scans often lack one or more modalities, which degrades segmentation performance. In this paper, we propose UniME (Uni-Encoder Meets Multi-Encoders), a two-stage heterogeneous method for brain tumor segmentation with missing modalities that reconciles the trade-offs among fine-grained structure capture, cross-modal complementarity modeling, and exploitation of available modalities. The idea is to decouple representation learning from segmentation via a two-stage heterogeneous architecture. Stage 1 pretrains a single ViT Uni-Encoder with masked image modeling to establish a unified representation robust to missing modalities. Stage 2 adds modality-specific CNN Multi-Encoders to extract high-resolution, multi-scale, fine-grained features. We fuse these features with the global representation to produce precise segmentations. Experiments on BraTS 2023 and BraTS 2024 show that UniME outperforms previous methods under incomplete multi-modal scenarios. The code is available at https://github.com/Hooorace-S/UniME
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。