将病理图像与空间转录组数据融合，构建更精准的细胞表征空间。
SPADE: Spatial Transcriptomics and Pathology Alignment Using a Mixture of Data Experts for an Expressive Latent Space
- 用多专家混合模型学习图像与基因表达的联合表征
- 在20个下游任务中少样本表现显著优于基线模型
- 适合研究肿瘤异质性或开发病理辅助诊断工具的人群
数字病理学和自监督深度学习的快速发展推动了多种疾病病理任务的基础模型发展。尽管多模态方法已出现，但全切片图像（WSI）与空间转录组（ST）的全面整合仍存在关键缺口，而这对于捕捉标准苏木精-伊红（H&E）染色无法体现的分子异质性至关重要。我们提出SPADE，一个将组织病理学与空间转录组数据融合的基础模型，通过统一框架引导图像表征学习，从而构建出受空间转录组信息启发的潜在空间。SPADE采用数据专家混合技术，通过两阶段基于对比学习的图像特征聚类生成专家，以学习共注册的WSI切片与基因表达谱的表示。在包含1000张样本的HEST-1k数据集上预训练后，SPADE在20个下游任务中表现出显著优于基线模型的少样本性能，验证了将形态与分子信息整合至统一潜在空间的有效性。代码与预训练权重已开源。
原文摘要 · Abstract (English)
The rapid growth of digital pathology and advances in self-supervised deep learning have enabled the development of foundational models for various pathology tasks across diverse diseases. While multimodal approaches integrating diverse data sources have emerged, a critical gap remains in the comprehensive integration of whole-slide images (WSIs) with spatial transcriptomics (ST), which is crucial for capturing critical molecular heterogeneity beyond standard hematoxylin & eosin (H&E) staining. We introduce SPADE, a foundation model that integrates histopathology with ST data to guide image representation learning within a unified framework, in effect creating an ST-informed latent space. SPADE leverages a mixture-of-data experts technique, where experts are created via two-stage imaging feature-space clustering using contrastive learning to learn representations of co-registered WSI patches and gene expression profiles. Pre-trained on the comprehensive HEST-1k dataset, SPADE is evaluated on 20 downstream tasks, demonstrating significantly superior few-shot performance compared to baseline models, highlighting the benefits of integrating morphological and molecular information into one latent space. Code and pretrained weights are available at https://github.com/uclabair/SPADE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。