用病理图像和临床描述生成真实基因表达谱，提升疾病预测准确率
GeMM-GAN: A Multimodal Generative Model Conditioned on Histopathology Images and Clinical Descriptions for Gene Expression Profile Generation
- 结合病理图像与临床文本，通过跨模态注意力生成基因表达条件向量
- 在TCGA数据集上使疾病分类准确率提升超11%，优于当前最优生成模型
- 适合需要合成基因数据的生物医学研究者，尤其关注隐私保护场景
生物医学研究日益依赖整合基因表达谱、医学影像和临床元数据等多模态数据。尽管医学影像和临床元数据在临床实践中广泛收集，基因表达数据却因严格隐私法规和高昂实验成本而难以普及。为此，我们提出GeMM-GAN，一种基于组织病理切片和临床元数据的新型生成对抗网络，用于合成真实可信的基因表达谱。该模型采用Transformer编码器处理图像块，并通过图像块与文本标记间的交叉注意力机制生成条件向量，引导生成模型输出具有生物学合理性的基因表达谱。我们在TCGA数据集上评估该方法，结果表明其性能超越标准生成模型，生成的基因表达谱更真实且功能意义更强，在下游疾病类型预测任务中准确率较当前最优生成模型提升超过11%。代码将公开于：https://github.com/francescapia/GeMM-GAN
原文摘要 · Abstract (English)
Biomedical research increasingly relies on integrating diverse data modalities, including gene expression profiles, medical images, and clinical metadata. While medical images and clinical metadata are routinely collected in clinical practice, gene expression data presents unique challenges for widespread research use, mainly due to stringent privacy regulations and costly laboratory experiments. To address these limitations, we present GeMM-GAN, a novel Generative Adversarial Network conditioned on histopathology tissue slides and clinical metadata, designed to synthesize realistic gene expression profiles. GeMM-GAN combines a Transformer Encoder for image patches with a final Cross Attention mechanism between patches and text tokens, producing a conditioning vector to guide a generative model in generating biologically coherent gene expression profiles. We evaluate our approach on the TCGA dataset and demonstrate that our framework outperforms standard generative models and generates more realistic and functionally meaningful gene expression profiles, improving by more than 11\% the accuracy on downstream disease type prediction compared to current state-of-the-art generative models. Code will be available at: https://github.com/francescapia/GeMM-GAN
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。