用双图注意力网络融合多模态肺结节数据,提升恶性预测准确率。
DGSAN: Dual-Graph Spatiotemporal Attention Network for Pulmonary Nodule Malignancy Prediction
- 构建跨模态与模态内双图结构,动态捕捉特征关系。
- 在两个数据集上超越现有方法,分类精度显著提升。
- 适用于医学影像分析、肺癌早期诊断研究者。
肺癌仍是全球癌症致死的首要原因。早期发现肺结节对提高患者生存率至关重要。尽管已有研究整合多模态与多时相信息,优于单一模态和单时间点方法,但融合方式仍局限于低效的向量拼接和简单互注意力机制,亟需更高效的多模态信息融合方案。为此,我们提出双图时空注意力网络(DGSAN),利用时间变化与多模态数据提升恶性预测准确性。方法包括:全局-局部特征编码器以捕捉结节的局部、全局及融合特征;双图构建方法将多模态特征组织为跨模态与模态内图结构;层级跨模态图融合模块实现特征精细化整合。同时,我们构建了新的多模态数据集NLST-cmst,为相关研究提供支持。大量实验在NLST-cmst与自建的CSTL衍生数据集上进行,结果表明DGSAN显著优于当前先进方法,在分类性能与计算效率方面均表现优异。
原文摘要 · Abstract (English)
Lung cancer continues to be the leading cause of cancer-related deaths globally. Early detection and diagnosis of pulmonary nodules are essential for improving patient survival rates. Although previous research has integrated multimodal and multi-temporal information, outperforming single modality and single time point, the fusion methods are limited to inefficient vector concatenation and simple mutual attention, highlighting the need for more effective multimodal information fusion. To address these challenges, we introduce a Dual-Graph Spatiotemporal Attention Network, which leverages temporal variations and multimodal data to enhance the accuracy of predictions. Our methodology involves developing a Global-Local Feature Encoder to better capture the local, global, and fused characteristics of pulmonary nodules. Additionally, a Dual-Graph Construction method organizes multimodal features into inter-modal and intra-modal graphs. Furthermore, a Hierarchical Cross-Modal Graph Fusion Module is introduced to refine feature integration. We also compiled a novel multimodal dataset named the NLST-cmst dataset as a comprehensive source of support for related research. Our extensive experiments, conducted on both the NLST-cmst and curated CSTL-derived datasets, demonstrate that our DGSAN significantly outperforms state-of-the-art methods in classifying pulmonary nodules with exceptional computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。