arXiv:2410.10547cs.CVcs.AI2024-10被引 9

融合手写图像与动态信号,用混合注意力模型提升阿尔茨海默病早期检测精度。

Hybrid Transformer for Early Alzheimer's Detection: Integration of Handwriting-Based 2D Images and 1D Signal Features

  • 设计双模态混合注意力网络,联合学习手写2D图像与1D信号特征。
  • 在DARWIN数据集上实现90.32%的F1分数,优于此前最优结果4.61%。
  • 适合关注多模态融合与神经退行性疾病早期诊断的研究者。

阿尔茨海默病(AD)是一种常见的神经退行性疾病,早期检测至关重要。手写变化常在疾病早期即出现,是一种无创且低成本的运动功能异常捕捉方式。现有基于手写的在线AD检测研究多依赖人工提取特征,并输入浅层机器学习模型;近年虽有采用1D-CNN或2D-CNN的深度学习方法,性能优于传统方案,但忽视了手写笔画2D空间模式与其1D动态特征之间的内在关联,难以充分捕捉手写数据的多模态特性。此外,Transformer模型在此任务中几乎未被探索。为此,本文提出一种新型AD检测方法,构建可学习的多模态混合注意力模型,同时融合2D手写图像与1D动态手写信号。模型通过门控机制结合相似性与差异性注意力,跨尺度融合信息以学习鲁棒特征。在DARWIN数据集任务8('L'书写)中,模型取得90.32%的F1分数和90.91%的准确率,分别较前人最佳提升4.61%与6.06%,达到当前最优水平。

原文摘要 · Abstract (English)

Alzheimer's Disease (AD) is a prevalent neurodegenerative condition where early detection is vital. Handwriting, often affected early in AD, offers a non-invasive and cost-effective way to capture subtle motor changes. State-of-the-art research on handwriting, mostly online, based AD detection has predominantly relied on manually extracted features, fed as input to shallow machine learning models. Some recent works have proposed deep learning (DL)-based models, either 1D-CNN or 2D-CNN architectures, with performance comparing favorably to handcrafted schemes. These approaches, however, overlook the intrinsic relationship between the 2D spatial patterns of handwriting strokes and their 1D dynamic characteristics, thus limiting their capacity to capture the multimodal nature of handwriting data. Moreover, the application of Transformer models remains basically unexplored. To address these limitations, we propose a novel approach for AD detection, consisting of a learnable multimodal hybrid attention model that integrates simultaneously 2D handwriting images with 1D dynamic handwriting signals. Our model leverages a gated mechanism to combine similarity and difference attention, blending the two modalities and learning robust features by incorporating information at different scales. Our model achieved state-of-the-art performance on the DARWIN dataset, with an F1-score of 90.32\% and accuracy of 90.91\% in Task 8 ('L' writing), surpassing the previous best by 4.61% and 6.06% respectively.

阿尔茨海默病多模态融合注意力机制手写分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。