arXiv:2510.21829cs.CV2025-10被引 1

用低秩Transformer与流模型解决多模态生存分析中的缺失数据问题。

A Flow Model with Low-Rank Transformers for Incomplete Multimodal Survival Analysis

  • 结合低秩Transformer与流模型,对缺失模态进行分布一致重建。
  • 在完整与不完整模态下均达当前最优,尤其在缺失场景下表现稳定。
  • 适合医学影像与基因组数据融合的生存预测研究者使用。

近年来,基于多模态医疗数据的生存分析备受关注。然而真实数据常存在模态缺失问题,部分患者因采集限制或系统故障导致某些模态信息缺失。现有方法通常通过深度神经网络直接从可观测模态推断缺失模态,但忽视了模态间的分布差异,造成重建结果不一致且不可靠。为此,我们提出一种新框架,将低秩Transformer与基于流的生成模型结合,实现鲁棒且灵活的多模态生存预测。具体地,将问题建模为基于全切片图像(WSIs)与基因组谱型的多实例生存分析。为实现不完整多模态生存分析,提出一种类别特定的流模型以实现跨模态分布对齐,在类别标签条件下建模并转换跨模态分布。借助归一化流的可逆结构与精确密度建模能力,有效构建缺失模态的一致潜在空间,提升重建数据与真实分布的一致性。最后,设计轻量级低秩Transformer架构,建模模态内依赖关系,同时通过低秩机制缓解高维模态融合中的过拟合问题。大量实验表明,该方法不仅在完整模态设置下达到领先性能,更在模态缺失场景中保持稳健且优越的准确性。

原文摘要 · Abstract (English)

In recent years, multimodal medical data-based survival analysis has attracted much attention. However, real-world datasets often suffer from the problem of incomplete modality, where some patient modality information is missing due to acquisition limitations or system failures. Existing methods typically infer missing modalities directly from observed ones using deep neural networks, but they often ignore the distributional discrepancy across modalities, resulting in inconsistent and unreliable modality reconstruction. To address these challenges, we propose a novel framework that combines a low-rank Transformer with a flow-based generative model for robust and flexible multimodal survival prediction. Specifically, we first formulate the concerned problem as incomplete multimodal survival analysis using the multi-instance representation of whole slide images (WSIs) and genomic profiles. To realize incomplete multimodal survival analysis, we propose a class-specific flow for cross-modal distribution alignment. Under the condition of class labels, we model and transform the cross-modal distribution. By virtue of the reversible structure and accurate density modeling capabilities of the normalizing flow model, the model can effectively construct a distribution-consistent latent space of the missing modality, thereby improving the consistency between the reconstructed data and the true distribution. Finally, we design a lightweight Transformer architecture to model intra-modal dependencies while alleviating the overfitting problem in high-dimensional modality fusion by virtue of the low-rank Transformer. Extensive experiments have demonstrated that our method not only achieves state-of-the-art performance under complete modality settings, but also maintains robust and superior accuracy under the incomplete modalities scenario.

生存分析多模态流模型低秩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。