用语音多模态数据提升抑郁症等心理问题的预测精度
Speech as a Multimodal Digital Phenotype for Multi-Task LLM-based Mental Health Prediction
- 将语音转文本、声学特征、声学生物标记三者融合建模
- 在青少年群体中实现70.8%的平衡准确率，优于单一模态方法
- 适合关注心理健康多任务诊断与长期动态分析的研究者
语音是一种无创的数字表型，可为心理健康状况提供重要信息，但通常被视为单一模态。本文提出将患者语音数据视为三模态多媒体数据源，用于抑郁检测。该研究探索基于大语言模型的架构，在整合语音生成文本、声学特征点和声学生物标记的多模态框架下进行语音抑郁预测。青少年抑郁症常伴随自杀意念和睡眠障碍等多种共病，为此我们引入多任务学习（MTL），同时预测抑郁、自杀意念和睡眠障碍。此外，提出一种纵向分析策略，建模多个临床交互中的时间变化，以全面理解病情进展。所提方法在“抑郁症早期预警”数据集上评估，平衡准确率达70.8%，高于各单模态、单任务及非纵向方法。
原文摘要 · Abstract (English)
Speech is a noninvasive digital phenotype that can offer valuable insights into mental health conditions, but it is often treated as a single modality. In contrast, we propose the treatment of patient speech data as a trimodal multimedia data source for depression detection. This study explores the potential of large language model-based architectures for speech-based depression prediction in a multimodal regime that integrates speech-derived text, acoustic landmarks, and vocal biomarkers. Adolescent depression presents a significant challenge and is often comorbid with multiple disorders, such as suicidal ideation and sleep disturbances. This presents an additional opportunity to integrate multi-task learning (MTL) into our study by simultaneously predicting depression, suicidal ideation, and sleep disturbances using the multimodal formulation. We also propose a longitudinal analysis strategy that models temporal changes across multiple clinical interactions, allowing for a comprehensive understanding of the conditions' progression. Our proposed approach, featuring trimodal, longitudinal MTL is evaluated on the Depression Early Warning dataset. It achieves a balanced accuracy of 70.8%, which is higher than each of the unimodal, single-task, and non-longitudinal methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。