融合影像、临床与放射组学数据,提升慢性肝病预后预测精度
TMI-CLNet: Triple-Modal Interaction Network for Chronic Liver Disease Prognosis From Imaging, Clinical, and Radiomic Data Fusion
- 设计跨模态注意力机制,挖掘三类数据间关联信息
- 在肝脏预后数据集上显著超越单模态与现有多模态方法
- 适合医疗人工智能、精准医学领域研究者参考
慢性肝病是全球重大健康挑战,准确的预后评估对个性化治疗至关重要。近期研究表明,整合计算机断层扫描影像、放射组学特征与临床信息等多模态数据,可提供更全面的预后信息。然而,各模态存在固有异质性,引入更多模态可能加剧异构数据融合的困难。现有多模态融合方法难以适应更丰富的医学模态,难以捕捉模态间关系。为此,我们提出三模态交互慢性肝病网络(TMI-CLNet)。具体地,设计了模态内聚合模块以消除模态内冗余,并构建三模态交叉注意力融合模块以提取跨模态信息;此外,设计三模态特征融合损失函数,对齐不同模态的特征表示。在肝脏预后数据集上的大量实验表明,该方法显著优于现有的先进单模态模型及其他多模态技术。
原文摘要 · Abstract (English)
Chronic liver disease represents a significant health challenge worldwide and accurate prognostic evaluations are essential for personalized treatment plans. Recent evidence suggests that integrating multimodal data, such as computed tomography imaging, radiomic features, and clinical information, can provide more comprehensive prognostic information. However, modalities have an inherent heterogeneity, and incorporating additional modalities may exacerbate the challenges of heterogeneous data fusion. Moreover, existing multimodal fusion methods often struggle to adapt to richer medical modalities, making it difficult to capture inter-modal relationships. To overcome these limitations, We present the Triple-Modal Interaction Chronic Liver Network (TMI-CLNet). Specifically, we develop an Intra-Modality Aggregation module and a Triple-Modal Cross-Attention Fusion module, which are designed to eliminate intra-modality redundancy and extract cross-modal information, respectively. Furthermore, we design a Triple-Modal Feature Fusion loss function to align feature representations across modalities. Extensive experiments on the liver prognosis dataset demonstrate that our approach significantly outperforms existing state-of-the-art unimodal models and other multi-modal techniques. Our code is available at https://github.com/Mysterwll/liver.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。