用三维对比学习对齐病理、基因和影像数据，提升胶质瘤分级与生存预测。
Trimodal Glioma Representation Alignment via Volumetric Contrastive Learning
- 通过三模态编码器分别处理病理切片、基因表达和3D MRI数据。
- 采用格拉姆对比损失对齐三者嵌入，提升跨模态一致性。
- 在132例患者数据上实现更优的分级与生存预测性能，适合多模态医学分析研究者。
胶质瘤分级与生存预测需整合不同空间与生物学尺度的异构信息：组织病理学描述组织形态，mRNA表达反映分子活动，磁共振成像提供肿瘤范围与影像异质性的非侵入性视图。现有预后模型通常仅融合两种模态，且对齐目标多为两两之间。本文提出GLORIA——一种针对胶质瘤组学-影像-病理对齐的新型三模态框架。GLORIA通过模态专用编码器处理全切片图像区域、基因表达谱和3D MRI体积，将其投影至共享潜在空间，并利用格拉姆对比损失衡量三模态嵌入所张成的体积进行对齐。对齐后的表示通过跨模态门控模块融合，并联合优化用于三分类胶质瘤分级与总生存预测。在包含132名患者（均具备三种模态）的匹配TCGA-GBM/LGG与BraTS21队列上评估，结果表明，在共享三模态测试集上，GLORIA在所有指标上均优于双模态WSI-mRNA基线。
原文摘要 · Abstract (English)
Glioma grading and survival prediction require the integration of heterogeneous information collected at different spatial and biological scales. Histopathology describes tissue morphology, mRNA expression captures molecular activity, and magnetic resonance imaging provides a non-invasive view of tumor extent and radiological heterogeneity. Existing glioma prognosis models often combine only two of these sources, while their alignment objectives remain mostly pairwise. This paper introduces GLORIA, a novel trimodal framework for GLioma Omics - Radiology - hIstopathology Alignment. GLORIA processes whole-slide image regions, gene-expression profiles, and 3D MRI volumes through modality-specific encoders, projects them into a shared latent space, and aligns them with a Gramian contrastive loss that measures the volume spanned by the three modality embeddings. The aligned representations are fused through a cross-modal gating module and optimized jointly for three-class glioma grading and overall survival prediction. We evaluate GLORIA on a matched TCGA-GBM/LGG and BraTS21 cohort, comprising 132 patients with all three modalities. On the shared trimodal test set, GLORIA improves over the bimodal WSI-mRNA baseline in all the metrics considered.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。