提出可适应任意模态组合的脑病变分割方法,显著提升模型泛化能力。
Towards Modality-Agnostic Continual Domain-Incremental Brain Lesion Segmentation
- 通过通道膨胀策略统一不同模态输入,支持任意模态组合
- 引入轻量级文本嵌入提供全局上下文,提升3D局部特征表达
- 采用精选样本重放机制,有效缓解无缓冲场景下的遗忘问题
多模态MRI中的脑病变分割通常假设模态集合固定或病理类型预先定义,导致现有模型难以跨队列和成像协议迁移。持续学习(CL)提供了自然解决方案,但当前方法要么限制最大模态配置,要么在无缓冲设置下出现严重遗忘。本文提出CLMU-Net,一种基于重放的3D脑病变分割持续学习框架,支持任意且可变的模态组合,无需事先知道最大模态集。一种概念简单但有效的通道膨胀策略将任意模态子集映射为统一的多通道表示,使单一模型可在多种数据集上运行。为增强固有的局部3D块特征,引入轻量级域条件文本嵌入,为每个训练样本提供全局模态-疾病上下文。通过包含原型与挑战性样本的紧凑缓冲区进行合理重放,进一步减少遗忘。在五个异构MRI脑数据集上的实验表明,CLMU-Net始终优于主流持续学习基线。特别地,该方法在异构模态条件下平均Dice得分提升≥18%,证明了灵活模态处理、针对性重放和全局上下文线索对持续医学图像分割的重要价值。代码已开源。
原文摘要 · Abstract (English)
Brain lesion segmentation from multi-modal MRI often assumes fixed modality sets or predefined pathologies, making existing models difficult to adapt across cohorts and imaging protocols. Continual learning (CL) offers a natural solution but current approaches either impose a maximum modality configuration or suffer from severe forgetting in buffer-free settings. We introduce CLMU-Net, a replay-based CL framework for 3D brain lesion segmentation that supports arbitrary and variable modality combinations without requiring prior knowledge of the maximum set. A conceptually simple yet effective channel-inflation strategy maps any modality subset into a unified multi-channel representation, enabling a single model to operate across diverse datasets. To enrich inherently local 3D patch features, we incorporate lightweight domain-conditioned textual embeddings that provide global modality-disease context for each training case. Forgetting is further reduced through principled replay using a compact buffer composed of both prototypical and challenging samples. Experiments on five heterogeneous MRI brain datasets demonstrate that CLMU-Net consistently outperforms popular CL baselines. Notably, our method yields an average Dice score improvement of $\geq$ 18\% while remaining robust under heterogeneous-modality conditions. These findings underscore the value of flexible modality handling, targeted replay, and global contextual cues for continual medical image segmentation. Our implementation is available at https://github.com/xmindflow/CLMU-Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。