用多模态深度学习分析语音和文本，提升抑郁症自动检测准确率。
Context-Aware Deep Learning for Multi Modal Depression Detection
- 融合预训练Transformer与主题建模增强文本特征，用1D CNN处理音频特征。
- 在音频和文本模态上分别达到当前最优性能，联合模型超越已有方法。
- 适合临床辅助诊断、心理健康研究者参考，代码开源可复现。
本研究聚焦于利用多模态机器学习（ML）从临床访谈中自动检测抑郁症。与现有成功方法不同，我们提出一种基于上下文感知的分析方法：（1）结合预训练Transformer与基于主题建模的数据增强处理文本数据；（2）采用深层1维卷积神经网络（1D CNN）建模声学特征。仿真结果表明，所提方法在训练多模态深度学习模型方面具有显著有效性。我们的1D CNN模型在音频模态上表现最佳，Transformer模型在文本模态上达到领先水平。将两者整合为多模态框架后，在联合设置下也优于现有最优方法。代码已公开于https://github.com/genandlam/multi-modal-depression-detection。
原文摘要 · Abstract (English)
In this study, we focus on automated approaches to detect depression from clinical interviews using multi-modal machine learning (ML). Our approach differentiates from other successful ML methods such as context-aware analysis through feature engineering and end-to-end deep neural networks for depression detection utilizing the Distress Analysis Interview Corpus. We propose a novel method that incorporates: (1) pre-trained Transformer combined with data augmentation based on topic modelling for textual data; and (2) deep 1D convolutional neural network (CNN) for acoustic feature modeling. The simulation results demonstrate the effectiveness of the proposed method for training multi-modal deep learning models. Our deep 1D CNN and Transformer models achieved state-of-the-art performance for audio and text modalities respectively. Combining them in a multi-modal framework also outperforms state-of-the-art for the combined setting. Code available at https://github.com/genandlam/multi-modal-depression-detection
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。