arXiv:2507.05591cs.AI2025-07被引 7

用多模态大模型实现可解释抑郁识别,效果领先。

MLlm-DR: Towards Explainable Depression Recognition with MultiModal Large Language Models

  • 结合小模型与轻量查询模块,融合语音视觉信息进行诊断。
  • 在CMDC和E-DAIC-WOZ数据集上达到最好性能。
  • 生成诊断理由,适合临床辅助决策场景。

自动抑郁诊断旨在分析访谈视频中的多模态信息以预测参与者的抑郁评分。以往研究常缺乏对评分依据的清晰解释,限制了其在临床实践中的应用。尽管大语言模型(LLM)为可解释诊断提供了可能,但现有能处理多模态数据的LLM未在访谈数据上训练,直接使用时诊断效果不佳。本文提出一种新型多模态大语言模型MLlm-DR,可理解多模态输入并支持可解释的抑郁诊断。MLlm-DR整合了一个小型LLM和一个轻量级查询模块(LQ-former)。小型LLM负责生成抑郁评分及评估理由;为增强其在领域任务上的逻辑推理能力同时保持实用性,我们构建了稳健的训练数据集对其进行微调。LQ-former则从语音和视觉数据中捕捉抑郁相关特征,提升模型处理多模态信息的能力,实现全面的抑郁诊断。本方法在两个基于访谈的基准数据集CMDC和E-DAIC-WOZ上达到最先进水平,证明了其有效性和优越性。

原文摘要 · Abstract (English)

Automated depression diagnosis aims to analyze multimodal information from interview videos to predict participants' depression scores. Previous studies often lack clear explanations of how these scores were determined, limiting their adoption in clinical practice. While the advent of LLMs provides a possible pathway for explainable depression diagnosis, current LLMs capable of processing multimodal data lack training on interview data, resulting in poor diagnostic performance when used directly. In this paper, we propose a novel multimodal large language model (MLlm-DR) that can understand multimodal information inputs and supports explainable depression diagnosis. MLlm-DR integrates a smaller LLMs and a lightweight query module (LQ-former). Specifically, the smaller LLMs is designed to generate depression scores and corresponding evaluation rationales. To enhance its logical reasoning for domain-specific tasks while maintaining practicality, we constructed a robust training dataset to fine-tune it. Meanwhile, the LQ-former captures depression-related features from speech and visual data, aiding the model's ability to process multimodal information, to achieve comprehensive depression diagnosis. Our approach achieves state-of-the-art results on two interview-based benchmark datasets, CMDC and E-DAIC-WOZ, demonstrating its effectiveness and superiority.

抑郁识别多模态可解释大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。