用Transformer统一处理缺失的癌症数据，提升生存预测准确率。
impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Prediction
- 通过掩码重建学习多模态间与内部关系，同步填补缺失数据。
- 在TCGA和BraTS数据集上达到最新最优效果，生存预测精度显著提升。
- 适合处理多源医疗数据缺失问题的研究者，尤其关注癌症预后建模。
多模态数据（如组学、医学影像和临床数据）不仅能提升预后模型性能，还能加深对疾病机制的理解并推动新疗法开发。然而，医疗数据复杂且常含缺失模态，有效处理缺失是训练多模态模型的关键。本文提出impuTMAE，一种基于Transformer的端到端方法，采用高效的多模态预训练策略，在学习模态间与模态内交互的同时，通过重建被掩码的特征块实现缺失模态的补全。模型在异构且不完整的数据上预训练，并在TCGA-GBM/LGG与BraTS数据集上微调，融合五类模态：遗传（DNAm、RNA-seq）、影像（MRI、WSI）与临床数据。通过在预训练阶段解决缺失问题并实现高效资源利用，impuTMAE超越现有方法，在胶质瘤患者生存预测任务中取得当前最优表现。代码已开源。
原文摘要 · Abstract (English)
The use of diverse modalities, such as omics, medical images, and clinical data can not only improve the performance of prognostic models but also deepen an understanding of disease mechanisms and facilitate the development of novel treatment approaches. However, medical data are complex, often incomplete, and contains missing modalities, making effective handling its crucial for training multimodal models. We introduce impuTMAE, a novel transformer-based end-to-end approach with an efficient multimodal pre-training strategy. It learns inter- and intra-modal interactions while simultaneously imputing missing modalities by reconstructing masked patches. Our model is pre-trained on heterogeneous, incomplete data and fine-tuned for glioma survival prediction using TCGA-GBM/LGG and BraTS datasets, integrating five modalities: genetic (DNAm, RNA-seq), imaging (MRI, WSI), and clinical data. By addressing missing data during pre-training and enabling efficient resource utilization, impuTMAE surpasses prior multimodal approaches, achieving state-of-the-art performance in glioma patient survival prediction. Our code is available at https://github.com/maryjis/mtcp
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。