用病理图和不完整基因数据做癌症原型，提升精准医疗效果
Interpretable Multimodal Cancer Prototyping with Whole Slide Images and Incompletely Paired Genomics
- 通过文本提示和原型加权实现生物可解释的癌症原型构建
- 在多个任务中优于现有方法，尤其在基因数据缺失时表现稳定
- 适合临床真实场景，可处理部分或完全缺失的基因信息
整合病理图像与基因组数据的多模态方法在精准肿瘤学中潜力巨大。但表型与基因型异质性会降低模态内表示质量，阻碍跨模态融合。此外，多数方法忽略临床中基因数据可能部分缺失或完全不可用的现实。本文提出一种灵活的多模态原型框架，用于整合全幻灯片图像与不完整基因组数据。包含四个核心组件：1）基于文本提示与原型加权的生物原型构建；2）样本级与分布级对齐的多视图对齐；3）捕捉共性和模态特异性信息的双边融合；4）缺失数据的语义基因组补全。大量实验表明，该方法在多个下游任务中持续优于现有先进方法。代码已开源：https://github.com/helenypzhang/Interpretable-Multimodal-Prototyping。
原文摘要 · Abstract (English)
Multimodal approaches that integrate histology and genomics hold strong potential for precision oncology. However, phenotypic and genotypic heterogeneity limits the quality of intra-modal representations and hinders effective inter-modal integration. Furthermore, most existing methods overlook real-world clinical scenarios where genomics may be partially missing or entirely unavailable. We propose a flexible multimodal prototyping framework to integrate whole slide images and incomplete genomics for precision oncology. Our approach has four key components: 1) Biological Prototyping using text prompting and prototype-wise weighting; 2) Multiview Alignment through sample- and distribution-wise alignments; 3) Bipartite Fusion to capture both shared and modality-specific information for multimodal fusion; and 4) Semantic Genomics Imputation to handle missing data. Extensive experiments demonstrate the consistent superiority of the proposed method compared to other state-of-the-art approaches on multiple downstream tasks. The code is available at https://github.com/helenypzhang/Interpretable-Multimodal-Prototyping.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。