用二值编码融合病理与基因数据，提升癌症诊断的效率与准确性。
Multimodal Learning for Scalable Representation of High-Dimensional Medical Data
- 自监督学习生成跨模态二值嵌入码，统一表示病理图像与基因数据。
- 肺癌诊断准确率85%-89%，优于单一模态模型。
- 适合需要高效检索临床病例的医疗AI研究者使用。
将人工智能与医疗数据结合正快速推动医学诊断发展，迈向精准医疗。然而，如何有效利用多模态数据——尤其是数字病理全切片图像（WSIs）和基因组测序数据——仍面临挑战，主要源于模态间固有异质性以及对可扩展、可解释框架的需求。现有诊断模型多基于单模态数据，忽略了能提供更丰富临床洞察的跨模态交互。本文提出MarbliX（Multimodal Association and Retrieval with Binary Latent Indexed matriX），一种自监督框架，将WSIs与免疫基因组特征嵌入紧凑且可扩展的二值代码（称为“monogram”）。通过跨模态三元组对比优化，该框架在统一潜在空间中捕捉高分辨率患者相似性，实现临床相关病例的高效检索，支持基于案例的推理。在肺癌中，MarbliX在所有评估指标上均达到85%-89%，显著优于组织病理学（69%-71%）和免疫基因组学（73%-76%）。在肾癌中，实值monogram表现最佳（F1: 80%-83%，准确率：87%-90%），二值monogram略低（F1: 78%-82%）。
原文摘要 · Abstract (English)
Integrating artificial intelligence (AI) with healthcare data is rapidly transforming medical diagnostics and driving progress toward precision medicine. However, effectively leveraging multimodal data, particularly digital pathology whole slide images (WSIs) and genomic sequencing, remains a significant challenge due to the intrinsic heterogeneity of these modalities and the need for scalable and interpretable frameworks. Existing diagnostic models typically operate on unimodal data, overlooking critical cross-modal interactions that can yield richer clinical insights. We introduce MarbliX (Multimodal Association and Retrieval with Binary Latent Indexed matriX), a self-supervised framework that learns to embed WSIs and immunogenomic profiles into compact, scalable binary codes, termed ``monogram.'' By optimizing a triplet contrastive objective across modalities, MarbliX captures high-resolution patient similarity in a unified latent space, enabling efficient retrieval of clinically relevant cases and facilitating case-based reasoning. \textcolor{black}{In lung cancer, MarbliX achieves 85-89\% across all evaluation metrics, outperforming histopathology (69-71\%) and immunogenomics (73-76\%). In kidney cancer, real-valued monograms yield the strongest performance (F1: 80-83\%, Accuracy: 87-90\%), with binary monograms slightly lower (F1: 78-82\%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。