用视觉变压器融合脑MRI与PET数据，提升阿尔茨海默病诊断准确率
DiaMond: Dementia Diagnosis with Multi-Modal Vision Transformers Using MRI and PET
- 引入自注意力与双模态注意力机制，协同融合MRI与PET信息
- 在阿尔茨海默病诊断中达92.4%平衡准确率，差分诊断达76.5%
- 适合临床辅助诊断研究者，尤其关注多模态医学影像分析
阿尔茨海默病（AD）和额颞叶痴呆（FTD）的诊断因症状重叠而复杂。尽管磁共振成像（MRI）和正电子发射断层扫描（PET）对诊断至关重要，但深度学习中多模态融合常面临挑战，性能往往不如单模态。此外，多模态方法在差分诊断中的潜力尚未充分探索。我们提出新框架DiaMond，基于视觉变压器有效整合MRI与PET数据。DiaMond采用自注意力与新型双注意力机制，协同融合两种模态，并引入多模态归一化以降低冗余依赖，从而提升性能。在多个数据集上显著优于现有方法：阿尔茨海默病诊断平衡准确率达92.4%，AD-MCI-CN分类为65.2%，AD与FTD差分诊断为76.5%。通过全面消融实验验证了其鲁棒性。代码已开源：https://github.com/ai-med/DiaMond。
原文摘要 · Abstract (English)
Diagnosing dementia, particularly for Alzheimer's Disease (AD) and frontotemporal dementia (FTD), is complex due to overlapping symptoms. While magnetic resonance imaging (MRI) and positron emission tomography (PET) data are critical for the diagnosis, integrating these modalities in deep learning faces challenges, often resulting in suboptimal performance compared to using single modalities. Moreover, the potential of multi-modal approaches in differential diagnosis, which holds significant clinical importance, remains largely unexplored. We propose a novel framework, DiaMond, to address these issues with vision Transformers to effectively integrate MRI and PET. DiaMond is equipped with self-attention and a novel bi-attention mechanism that synergistically combine MRI and PET, alongside a multi-modal normalization to reduce redundant dependency, thereby boosting the performance. DiaMond significantly outperforms existing multi-modal methods across various datasets, achieving a balanced accuracy of 92.4% in AD diagnosis, 65.2% for AD-MCI-CN classification, and 76.5% in differential diagnosis of AD and FTD. We also validated the robustness of DiaMond in a comprehensive ablation study. The code is available at https://github.com/ai-med/DiaMond.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。