用病理图像重建基因组信息，实现无测序癌症生存预测
VGAT: A Cancer Survival Analysis Framework Transitioning from Generative Visual Question Answering to Genomic Reconstruction
- 结合视觉问答技术从病理图中推断基因组特征
- 在五个TCGA数据集上超越现有仅用图像的方法
- 适合资源有限地区临床应用的癌症预后分析
将病理图像与基因组序列融合用于癌症生存分析虽具潜力，但在资源匮乏地区受限于基因测序获取困难。为仅使用全切片图像（WSI）实现生存预测，我们提出视觉-基因组问答引导变压器（VGAT），融合视觉问答技术以重建基因组模态。通过借鉴VQA的文本特征提取方法，获得稳定基因组表示，克服原始基因组数据的高维难题。同时，基于聚类的视觉提示模块选择性增强判别性WSI区域，减少非目标区域噪声影响。在五个TCGA数据集上评估显示，VGAT优于现有仅使用WSI的方法，证明了无需测序即可实现基因组指导推理的可行性。该方法弥合了多模态研究与资源受限环境下的临床可行性之间的鸿沟。
原文摘要 · Abstract (English)
Multimodal learning combining pathology images and genomic sequences enhances cancer survival analysis but faces clinical implementation barriers due to limited access to genomic sequencing in under-resourced regions. To enable survival prediction using only whole-slide images (WSI), we propose the Visual-Genomic Answering-Guided Transformer (VGAT), a framework integrating Visual Question Answering (VQA) techniques for genomic modality reconstruction. By adapting VQA's text feature extraction approach, we derive stable genomic representations that circumvent dimensionality challenges in raw genomic data. Simultaneously, a cluster-based visual prompt module selectively enhances discriminative WSI patches, addressing noise from unfiltered image regions. Evaluated across five TCGA datasets, VGAT outperforms existing WSI-only methods, demonstrating the viability of genomic-informed inference without sequencing. This approach bridges multimodal research and clinical feasibility in resource-constrained settings. The code link is https://github.com/CZZZZZZZZZZZZZZZZZ/VGAT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。