解决脑肿瘤分割中多模态磁共振数据缺失问题,提升不完整数据下的分割精度。
MGML: A Plug-and-Play Meta-Guided Multi-Modal Learning Framework for Incomplete Multimodal Brain Tumor Segmentation
- 用元参数自适应融合不同模态信息,动态生成软标签指导融合。
- 在BraTS2020上,全肿瘤分割平均Dice达87.55,优于已有方法。
- 无需修改原模型结构,可即插即用,适合临床实际场景应用。
利用磁共振成像(MRI)的多模态信息对脑肿瘤病灶分割至关重要,但临床上多模态数据常不完整,难以充分使用。为此,我们提出一种新的元引导多模态学习框架(MGML),包含元参数自适应模态融合与一致性正则化模块。Meta-AMF通过根据可用模态生成自适应软标签,显式促进更一致的多模态融合;一致性正则化模块增强分割性能并提升整体鲁棒性。该方法不改变原始模型结构,可无缝集成至训练流程实现端到端优化。在公开的BraTS2020和BraTS2023数据集上进行大量实验,结果表明:在十五种缺失模态组合下,相较于往年多个先进方法,本方法在BraTS2020上对全肿瘤(WT)、肿瘤核心(TC)、增强肿瘤(ET)的平均Dice分别达到87.55、79.36和62.67,显著提升。代码已开源:https://github.com/worldlikerr/MGML。
原文摘要 · Abstract (English)
Leveraging multimodal information from Magnetic Resonance Imaging (MRI) plays a vital role in lesion segmentation, especially for brain tumors. However, in clinical practice, multimodal MRI data are often incomplete, making it challenging to fully utilize the available information. Therefore, maximizing the utilization of this incomplete multimodal information presents a crucial research challenge. We present a novel meta-guided multi-modal learning (MGML) framework that comprises two components: meta-parameterized adaptive modality fusion and consistency regularization module. The meta-parameterized adaptive modality fusion (Meta-AMF) enables the model to effectively integrate information from multiple modalities under varying input conditions. By generating adaptive soft-label supervision signals based on the available modalities, Meta-AMF explicitly promotes more coherent multimodal fusion. In addition, the consistency regularization module enhances segmentation performance and implicitly reinforces the robustness and generalization of the overall framework. Notably, our approach does not alter the original model architecture and can be conveniently integrated into the training pipeline for end-to-end model optimization. We conducted extensive experiments on the public BraTS2020 and BraTS2023 datasets. Compared to multiple state-of-the-art methods from previous years, our method achieved superior performance. On BraTS2020, for the average Dice scores across fifteen missing modality combinations, building upon the baseline, our method obtained scores of 87.55, 79.36, and 62.67 for the whole tumor (WT), the tumor core (TC), and the enhancing tumor (ET), respectively. We have made our source code publicly available at https://github.com/worldlikerr/MGML.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。