用多模态注意力融合脑影像与认知数据,提升阿尔茨海默病早期诊断准确率
4D Multimodal Co-attention Fusion Network with Latent Contrastive Alignment for Alzheimer's Diagnosis
- 设计多片块对比损失,精准对齐功能与结构脑影像的时空特征
- 在ADNI数据集上达到91.3%准确率,显著优于现有方法
- 无需预设模态优先级,自动发现跨模态融合模式,适合临床辅助诊断
多模态神经影像可提供人类大脑组织结构与疾病动态的互补信息。近期研究显示,通过协同整合结构磁共振(sMRI)、功能磁共振(fMRI)与行为认知评分等表格式生物标志物数据,可提升阿尔茨海默病(AD)的诊断敏感性。然而,不同模态间固有的异质性(如4D时空fMRI动态与3D解剖sMRI结构)给判别性特征融合带来挑战。为此,我们提出M2M-AlignNet:一种基于几何感知的多模态共注意力网络,用于sMRI与fMRI的早期AD诊断。核心是多片块到多片块(M2M)对比损失函数,通过几何加权片块对应关系量化并减少表示差异,显式对齐不同脑区的功能成分与其解剖结构基础,且无需一一对应约束。此外,提出以潜在表示为查询的共注意力模块,自主发现融合模式,避免模态优先偏差并最小化特征冗余。我们在多个数据集上进行了广泛实验,验证了该方法的有效性,并强调了fMRI与sMRI之间的对应关系作为AD生物标志物的重要性。
原文摘要 · Abstract (English)
Multimodal neuroimaging provides complementary structural and functional insights into both human brain organization and disease-related dynamics. Recent studies demonstrate enhanced diagnostic sensitivity for Alzheimer's disease (AD) through synergistic integration of neuroimaging data (e.g., sMRI, fMRI) with behavioral cognitive scores tabular data biomarkers. However, the intrinsic heterogeneity across modalities (e.g., 4D spatiotemporal fMRI dynamics vs. 3D anatomical sMRI structure) presents critical challenges for discriminative feature fusion. To bridge this gap, we propose M2M-AlignNet: a geometry-aware multimodal co-attention network with latent alignment for early AD diagnosis using sMRI and fMRI. At the core of our approach is a multi-patch-to-multi-patch (M2M) contrastive loss function that quantifies and reduces representational discrepancies via geometry-weighted patch correspondence, explicitly aligning fMRI components across brain regions with their sMRI structural substrates without one-to-one constraints. Additionally, we propose a latent-as-query co-attention module to autonomously discover fusion patterns, circumventing modality prioritization biases while minimizing feature redundancy. We conduct extensive experiments to confirm the effectiveness of our method and highlight the correspondance between fMRI and sMRI as AD biomarkers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。