用大模型统一评估精神疾病,覆盖全生命周期多语言数据。
Foundation Model-based Evaluation of Neuropsychiatric Disorders: A Lifespan-Inclusive, Multi-Modal, and Multi-Lingual Study
- 构建多模态融合框架FEND,整合语音与文本信息
- 跨语言数据验证显示阿尔茨海默病和抑郁检测效果好,自闭症较差
- 揭示模态不平衡问题,适合做公平对比的基准研究
神经精神障碍(如阿尔茨海默病、抑郁症、自闭症谱系障碍)常伴随语言与语音异常,可作为早期生物标志物。尽管多模态方法前景广阔,但跨语言泛化能力弱、缺乏统一评估框架仍是主要挑战。为此,我们提出FEND(Foundation model-based Evaluation of Neuropsychiatric Disorders),一个融合语音与文本的多模态框架,用于全生命周期内对AD、抑郁、ASD的检测。基于13个涵盖英语、中文、希腊语、法语、荷兰语的多语言数据集,系统评估了多模态融合性能。结果显示,多模态融合在阿尔茨海默病和抑郁症检测中表现优异,但在自闭症中因数据异质性而表现不佳。同时发现模态不平衡普遍存在,多模态融合未能超越最佳单模态模型。跨语料库实验表明,在任务与语言一致时表现稳健,但在多语言及任务异构场景下性能明显下降。FEND提供了广泛基准与影响因素分析,推动自动化、全生命周期、多语言精神疾病评估的发展。我们鼓励研究者采用FEND框架进行公平比较与可复现研究。
原文摘要 · Abstract (English)
Neuropsychiatric disorders, such as Alzheimer's disease (AD), depression, and autism spectrum disorder (ASD), are characterized by linguistic and acoustic abnormalities, offering potential biomarkers for early detection. Despite the promise of multi-modal approaches, challenges like multi-lingual generalization and the absence of a unified evaluation framework persist. To address these gaps, we propose FEND (Foundation model-based Evaluation of Neuropsychiatric Disorders), a comprehensive multi-modal framework integrating speech and text modalities for detecting AD, depression, and ASD across the lifespan. Leveraging 13 multi-lingual datasets spanning English, Chinese, Greek, French, and Dutch, we systematically evaluate multi-modal fusion performance. Our results show that multi-modal fusion excels in AD and depression detection but underperforms in ASD due to dataset heterogeneity. We also identify modality imbalance as a prevalent issue, where multi-modal fusion fails to surpass the best mono-modal models. Cross-corpus experiments reveal robust performance in task- and language-consistent scenarios but noticeable degradation in multi-lingual and task-heterogeneous settings. By providing extensive benchmarks and a detailed analysis of performance-influencing factors, FEND advances the field of automated, lifespan-inclusive, and multi-lingual neuropsychiatric disorder assessment. We encourage researchers to adopt the FEND framework for fair comparisons and reproducible research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。