针对脑肿瘤分割中模态缺失问题,提出知识蒸馏框架提升模型鲁棒性。
No Modality Left Behind: Adapting to Missing Modalities via Knowledge Distillation for Brain Tumor Segmentation
- 通过图引导自适应精炼模块建模跨模态语义关联,增强缺失模态适应能力。
- 在单模态和弱模态配置下,分割精度显著优于现有方法,尤其在BraTS 2018/2024上表现突出。
- 适合临床实际中模态不全场景,为模型选型与未来研究提供实证指导。
准确的脑肿瘤分割对术前评估和个性化治疗至关重要。多模态MRI因能捕捉不同序列间的互补肿瘤特征而被广泛应用。然而,临床实践中模态缺失常见,限制了依赖完整输入的深度学习方法的鲁棒性和泛化能力,尤其在非主导模态组合下。为此,我们提出AdaMM,一种专为模态缺失场景设计的多模态脑肿瘤分割框架,核心为知识蒸馏,包含三个协同模块:图引导自适应精炼模块显式建模通用特征与模态特异性特征间的语义关联,提升对模态缺失的适应性;双向瓶颈蒸馏模块通过全局风格匹配与对抗特征对齐,将教师模型的结构与纹理知识传递给学生模型;病灶存在引导可靠性模块通过辅助分类任务预测病灶类型先验概率,有效抑制不完整输入下的假阳性。在Pretreat-MetsToBrain-Masks与BraTS 2018、2024数据集上的大量实验表明,AdaMM持续优于现有方法,在单模态与弱模态配置下展现出更优的分割精度与鲁棒性。此外,我们系统评估了六类缺失模态策略,验证了知识蒸馏的优势,并为方法选择与未来研究提供实践指导。源代码已公开于https://github.com/Quanato607/AdaMM。
原文摘要 · Abstract (English)
Accurate brain tumor segmentation is essential for preoperative evaluation and personalized treatment. Multi-modal MRI is widely used due to its ability to capture complementary tumor features across different sequences. However, in clinical practice, missing modalities are common, limiting the robustness and generalizability of existing deep learning methods that rely on complete inputs, especially under non-dominant modality combinations. To address this, we propose AdaMM, a multi-modal brain tumor segmentation framework tailored for missing-modality scenarios, centered on knowledge distillation and composed of three synergistic modules. The Graph-guided Adaptive Refinement Module explicitly models semantic associations between generalizable and modality-specific features, enhancing adaptability to modality absence. The Bi-Bottleneck Distillation Module transfers structural and textural knowledge from teacher to student models via global style matching and adversarial feature alignment. The Lesion-Presence-Guided Reliability Module predicts prior probabilities of lesion types through an auxiliary classification task, effectively suppressing false positives under incomplete inputs. Extensive experiments on the Pretreat-MetsToBrain-Masks and BraTS 2018, 2024 datasets demonstrate that AdaMM consistently outperforms existing methods, exhibiting superior segmentation accuracy and robustness, particularly in single-modality and weak-modality configurations. In addition, we conduct a systematic evaluation of six categories of missing-modality strategies, supporting the superiority of knowledge distillation and offering practical guidance for method selection and future research. Our source code is available at https://github.com/Quanato607/AdaMM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。