用大模型融合语音与心理学知识，提升抑郁识别准确率
Large Language Models for Depression Recognition in Spoken Language Integrating Psychological Knowledge
- 用Wav2Vec提取语音特征，接入大模型进行多模态分析
- 引入心理问答知识库，使模型诊断误差降低至MAE=0.417、RMSE=0.531
- 首个结合心理学先验的语音抑郁检测大模型，适合临床辅助诊断
抑郁症日益受到公众关注和人工智能研究重视。尽管深度神经网络已被用于识别，但实际效果有限。大语言模型具备潜力，但需领域微调且难以处理非文本线索。因抑郁常通过语调与行为表现，仅依赖语言不充分，且缺乏心理学知识会降低诊断精度。为此，我们首次在DAIC-WOZ数据集上应用大语言模型进行多模态抑郁检测。采用预训练模型Wav2Vec提取音频特征，并映射至文本型大模型进一步处理。同时提出一种新策略，通过问答集注入权威心理学知识以增强诊断能力。实验显示，该方法在均方误差（MAE）和均方根误差（RMSE）上显著优于原始论文基准。代码已开源。
原文摘要 · Abstract (English)
Depression is a growing concern gaining attention in both public discourse and AI research. While deep neural networks (DNNs) have been used for recognition, they still lack real-world effectiveness. Large language models (LLMs) show strong potential but require domain-specific fine-tuning and struggle with non-textual cues. Since depression is often expressed through vocal tone and behaviour rather than explicit text, relying on language alone is insufficient. Diagnostic accuracy also suffers without incorporating psychological expertise. To address these limitations, we present, to the best of our knowledge, the first application of LLMs to multimodal depression detection using the DAIC-WOZ dataset. We extract the audio features using the pre-trained model Wav2Vec, and mapped it to text-based LLMs for further processing. We also propose a novel strategy for incorporating psychological knowledge into LLMs to enhance diagnostic performance, specifically using a question and answer set to grant authorised knowledge to LLMs. Our approach yields a notable improvement in both Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) compared to a base score proposed by the related original paper. The codes are available at https://github.com/myxp-lyp/Depression-detection.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。