通过模态对齐与保留,提升病理与转录组数据融合的癌症表征学习效果。
MIRROR: Multi-Modal Pathological Self-Supervised Representation Learning via Modality Alignment and Retention
- 分模态编码+对齐模块,融合组织形态与基因表达特征。
- 在TCGA数据集上实现更优的癌症分型与生存预测性能。
- 适合需要多模态融合的肿瘤诊断与生物标志物研究者。
组织病理学和转录组学是肿瘤学中的基础模态,分别反映疾病的形态与分子特征。多模态自监督学习通过整合多种数据源,在学习病理表征方面展现出巨大潜力。传统方法主要关注模态对齐,却忽视了保持各模态特有的结构信息。然而,组织病理学与转录组学具有显著异质性,提供互补但独立的视角:前者揭示组织架构与细胞拓扑,后者通过基因表达模式刻画分子特征。这种内在差异使得两者对齐的同时维持模态特异性成为关键挑战。为此,我们提出MIRROR,一种新型多模态表征学习方法,旨在同时实现模态对齐与保留。MIRROR采用专用编码器提取各模态的全面特征,并通过模态对齐模块实现表型模式与分子谱型的无缝融合;同时,模态保留模块保护各模态独特属性,风格聚类模块通过建模并对齐聚类空间内的稳定病理特征,减少冗余、增强疾病相关信号。在TCGA队列上的广泛评估显示,MIRROR在癌症分型与生存分析任务中表现优异,验证了其构建综合性肿瘤特征表征的有效性,有助于提升癌症诊断能力。
原文摘要 · Abstract (English)
Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease. Multi-modal self-supervised learning has demonstrated remarkable potential in learning pathological representations by integrating diverse data sources. Conventional multi-modal integration methods primarily emphasize modality alignment, while paying insufficient attention to retaining the modality-specific structures. However, unlike conventional scenarios where multi-modal inputs share highly overlapping features, histopathology and transcriptomics exhibit pronounced heterogeneity, offering orthogonal yet complementary insights. Histopathology provides morphological and spatial context, elucidating tissue architecture and cellular topology, whereas transcriptomics delineates molecular signatures through gene expression patterns. This inherent disparity introduces a major challenge in aligning them while maintaining modality-specific fidelity. To address these challenges, we present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention. MIRROR employs dedicated encoders to extract comprehensive features for each modality, which is further complemented by a modality alignment module to achieve seamless integration between phenotype patterns and molecular profiles. Furthermore, a modality retention module safeguards unique attributes from each modality, while a style clustering module mitigates redundancy and enhances disease-relevant information by modeling and aligning consistent pathological signatures within a clustering space. Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR's superior performance, demonstrating its effectiveness in constructing comprehensive oncological feature representations and benefiting the cancer diagnosis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。