一个可自适应多种脑部MRI序列的通用模型，支持缺失序列补全。
A Foundation Model for Brain MRI with Dynamic Modality Integration
- 用可学习模态嵌入和条件归一化实现多序列统一编码
- 在约6万张多中心MRI上自监督训练，支持模态缺失时的重建
- 适用于肿瘤分割、病灶分类等任务，适合医疗影像研究者
我们提出一种用于脑部MRI的基础模型，能够处理不同成像序列的组合。该模型采用单一编码器，结合可学习的模态嵌入、条件层归一化及考虑缺失模态的掩码自编码目标。引入方差-协方差正则化以稳定特征学习并提升表示多样性。此设计避免了为每种模态单独建模的需求，使网络在部分序列缺失或未见时仍可自适应。模型在约6万张多中心MRI上通过自监督重建与模态补全进行训练，学习灵活表征。可学习的模态嵌入引导特征提取，使编码器能根据输入动态调整。我们计划在脑肿瘤和多发性硬化症分割、病灶分类等任务中评估其表现，涵盖多种模态配置。初步结果表明方法可行，后续将开展更详细实验验证。所有代码与预训练模型已公开于https://github.com/BrainFM/brainfm。
原文摘要 · Abstract (English)
We present a foundation model for brain MRI that can work with different combinations of imaging sequences. The model uses one encoder with learnable modality embeddings, conditional layer normalization, and a masked autoencoding objective that accounts for missing modalities. A variance-covariance regularizer is applied to stabilize feature learning and improve representation diversity. This design removes the need for separate models for each modality and allows the network to adapt when some sequences are missing or unseen. It is trained on about 60,000 multi-center MRIs using self-supervised reconstruction and modality imputation to learn flexible representations. A learnable modality embedding guides feature extraction so the encoder can adjust to different inputs. We describe our planned evaluation on brain tumor and multiple sclerosis segmentation, as well as lesion classification, under various modality settings. Preliminary results show that the method works feasibly, and further experiments are planned to study its performance in more detail. All code and pretrained models are available at https://github.com/BrainFM/brainfm
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。