用跨模态注意力融合脑结构与功能数据，提升精神疾病诊断准确率
Unified Cross-Modal Attention-Mixer Based Structural-Functional Connectomics Fusion for Neuropsychiatric Disorder Diagnosis
- 通过交叉注意力和MLP-Mixer融合结构与功能脑网络特征
- 在两个临床数据集上实现更优分类性能，提升诊断可靠性
- 适合脑科学与医学人工智能研究者参考
理解大脑的结构与功能机制是神经科学长期关注的重点，尤其在精神疾病如精神分裂症（SZ）的诊疗中。然而，传统多模态深度学习方法未能充分挖掘结构与功能连接组数据的互补性，限制了诊断效果。为此，本文提出ConneX，一种结合交叉注意力机制与MLP-Mixer的多模态融合方法，实现精细化特征融合。首先使用模态专用的图神经网络（GNN）提取各模态特征表示，随后引入统一的跨模态注意力网络，捕获模态内与模态间交互关系；MLP-Mixer层进一步优化全局与局部特征，利用高阶依赖关系，在多头联合损失下实现端到端分类。在两个不同临床数据集上的广泛评估表明，该框架显著提升诊断性能，展现出良好的鲁棒性。
原文摘要 · Abstract (English)
Gaining insights into the structural and functional mechanisms of the brain has been a longstanding focus in neuroscience research, particularly in the context of understanding and treating neuropsychiatric disorders such as Schizophrenia (SZ). Nevertheless, most of the traditional multimodal deep learning approaches fail to fully leverage the complementary characteristics of structural and functional connectomics data to enhance diagnostic performance. To address this issue, we proposed ConneX, a multimodal fusion method that integrates cross-attention mechanism and multilayer perceptron (MLP)-Mixer for refined feature fusion. Modality-specific backbone graph neural networks (GNNs) were firstly employed to obtain feature representation for each modality. A unified cross-modal attention network was then introduced to fuse these embeddings by capturing intra- and inter-modal interactions, while MLP-Mixer layers refined global and local features, leveraging higher-order dependencies for end-to-end classification with a multi-head joint loss. Extensive evaluations demonstrated improved performance on two distinct clinical datasets, highlighting the robustness of our proposed framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。