融合文本、音频和面部信号,实现抑郁与创伤后应激障碍的分级诊断。
Tri-Modal Severity Fused Diagnosis across Depression and Post-traumatic Stress Disorders
- 三模态数据同步融合,输出抑郁(5级)与PTSD(3级)的严重程度评分。
- 在DAIC数据集上,模型准确率和加权F1达最优,噪声下仍保持稳健。
- 适合临床辅助决策,提供可解释性特征贡献,支持医生参与评估。
抑郁与创伤后应激障碍(PTSD)常共病且症状重叠,自动化评估多为二分类且针对单一疾病,难以满足临床需求。本文提出统一的三模态情感严重度框架,同步融合访谈文本(句级Transformer嵌入)、音频(对数梅尔频谱及导数)与面部信号(动作单元、注视、头部姿态)。通过校准的晚期融合分类器,输出抑郁(PHQ-8,5类)与PTSD(3类)的分级严重度,并提供特征级归因。在基于DAIC语料库的分层交叉验证中,该方法优于单模态及消融基线。融合模型在准确率与加权F1上匹配最强单模态基线,同时提升决策曲线效用并增强对噪声或缺失模态的鲁棒性。对PTSD而言,融合显著降低回归误差并提高类别一致性。错误主要集中在相邻严重等级间,极端等级识别可靠。消融实验表明:文本对抑郁严重度贡献最大,音频与面部线索对PTSD至关重要,归因结果与语言及行为标志一致。本方法支持可复现评估与医生在环决策支持。
原文摘要 · Abstract (English)
Depression and post traumatic stress disorder (PTSD) often co-occur with connected symptoms, complicating automated assessment, which is often binary and disorder specific. Clinically useful diagnosis needs severity aware cross disorder estimates and decision support explanations. Our unified tri modal affective severity framework synchronizes and fuses interview text with sentence level transformer embeddings, audio with log Mel statistics with deltas, and facial signals with action units, gaze, head and pose descriptors to output graded severities for diagnosing both depression (PHQ-8; 5 classes) and PTSD (3 classes). Standardized features are fused via a calibrated late fusion classifier, yielding per disorder probabilities and feature-level attributions. This severity aware tri-modal affective fusion approach is demoed on multi disorder concurrent depression and PTSD assessment. Stratified cross validation on DAIC derived corpora outperforms unimodal/ablation baselines. The fused model matches the strongest unimodal baseline on accuracy and weighted F1, while improving decision curve utility and robustness under noisy or missing modalities. For PTSD specifically, fusion reduces regression error and improves class concordance. Errors cluster between adjacent severities; extreme classes are identified reliably. Ablations show text contributes most to depression severity, audio and facial cues are critical for PTSD, whereas attributions align with linguistic and behavioral markers. Our approach offers reproducible evaluation and clinician in the loop support for affective clinical decision making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。