用动态图神经微分方程建模对话情绪时序,提升识别准确率。
Dynamic Graph Neural ODE Network for Multi-modal Emotion Recognition in Conversation
- 引入自适应MixHop与图ODE,捕捉情绪随时间的连续变化
- 在两个公开数据集上超越基线模型,准确率显著提升
- 适合研究多模态情感分析与动态图神经网络的学者
对话中的多模态情感识别(MERC)旨在通过融合音频、文本、视频等多源信息识别和分类人类情感状态。现有方法多采用图卷积网络(GCN)提升性能,但易过拟合,且难以捕捉说话人情绪的时序依赖。为此,本文提出动态图神经常微分方程网络(DGODE),通过建模情绪的动态演化来捕捉时序依赖,并有效缓解GCN的过拟合问题。技术上,DGODE采用自适应MixHop机制增强GCN泛化能力,结合图ODE演化网络刻画节点表示在时间上的连续动态,从而捕获时序特征。在两个公开多模态情感识别数据集上的大量实验表明,所提模型性能优于多种基线方法。此外,该模型还能缓解过平滑问题,支持构建深层GCN网络。
原文摘要 · Abstract (English)
Multimodal emotion recognition in conversation (MERC) refers to identifying and classifying human emotional states by combining data from multiple different modalities (e.g., audio, images, text, video, etc.). Most existing multimodal emotion recognition methods use GCN to improve performance, but existing GCN methods are prone to overfitting and cannot capture the temporal dependency of the speaker's emotions. To address the above problems, we propose a Dynamic Graph Neural Ordinary Differential Equation Network (DGODE) for MERC, which combines the dynamic changes of emotions to capture the temporal dependency of speakers' emotions, and effectively alleviates the overfitting problem of GCNs. Technically, the key idea of DGODE is to utilize an adaptive mixhop mechanism to improve the generalization ability of GCNs and use the graph ODE evolution network to characterize the continuous dynamics of node representations over time and capture temporal dependencies. Extensive experiments on two publicly available multimodal emotion recognition datasets demonstrate that the proposed DGODE model has superior performance compared to various baselines. Furthermore, the proposed DGODE can also alleviate the over-smoothing problem, thereby enabling the construction of a deep GCN network.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。