动态融合与语义扫描,提升多模态生存分析精度
AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis

- 动态调节多模态交互强度,适应不同患者和区域的诊断重要性变化
- 在五个TCGA队列中优于现有方法,显著提升生存预测性能
- 适合医学影像与基因组数据融合分析的研究者使用
利用全切片图像(WSIs)和基因组谱进行多模态生存分析对癌症预后至关重要。近年来,状态空间模型如Mamba在序列建模中表现优异,但将其应用于复杂多模态任务仍面临两大挑战:其一,传统融合策略假设模态间交互强度恒定,忽略了不同患者和局部区域中各模态诊断重要性的动态变化;其二,标准Mamba架构沿预设物理路径处理令牌,导致空间分散的医学特征语义不连续,加剧长程衰减。为此,我们提出AdaSurvMamba,一种新型自适应框架。该框架包含双尺度重要性感知重构(DSIR)模块,从序列与令牌层面评估诊断重要性并重构输入表示,实现动态交叉模态调制。同时引入语义聚合扫描(SAS)模块,通过共享原型池将离散令牌重新组织为语义连续序列,基于全局模态上下文与语义先验显式调节状态转移步长,自适应控制信息吸收速率。在五个TCGA队列上的实验表明,该方法持续优于现有方法。代码已开源。
原文摘要 · Abstract (English)
Multimodal survival analysis utilizing whole slide images (WSIs) and genomic profiles is fundamental for cancer prognosis. Recently, state-space models like Mamba have emerged as powerful tools for sequence modeling. However, translating this success to complex multimodal tasks is hindered by two critical limitations. First, conventional fusion strategies assume a static multimodal interaction strength, ignoring the fluctuating diagnostic importance of each modality across different patients and local regions. Second, the standard Mamba architecture processes tokens along predefined physical paths. This rigid scanning disrupts the semantic continuity of spatially scattered medical features and exacerbates long-range decay. To address these challenges, we introduce AdaSurvMamba as a novel adaptive framework for multimodal survival analysis. The framework features a Dual-Scale Importance-Aware Reconstruction (DSIR) module to dynamically modulate cross-modal interaction strength. It evaluates diagnostic importance at both the sequence and token levels to reconstruct the input representations. Furthermore, we propose a Semantic Aggregation Scanning (SAS) module to overcome contextual fragmentation. The SAS module dynamically reorganizes discrete tokens into semantically continuous sequences via a shared prototype pool. It explicitly modulates the state transition step size using global modality context and semantic priors to adaptively control the information absorption rate. Experiments across five TCGA cohorts demonstrate consistent gains over existing methods. Code is available at https://github.com/zjlGO/AdaSurvMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。