arXiv:2502.13954cs.CLcs.LG2025-02被引 2

通过概率建模解耦情绪分布,提升多模态情绪识别的不确定性感知能力。

Latent Distribution Decoupling: A Probabilistic Framework for Uncertainty-Aware Multimodal Emotion Recognition

  • 在情绪潜空间中设计对比解耦分布机制,分离语义特征与不确定性。
  • 在CMU-MOSEI和M³ED上达到最新最优性能,验证了不确定性建模的有效性。
  • 适合关注多模态融合中噪声鲁棒性的研究者或应用开发者。

多模态多标签情绪识别(MMER)旨在从多模态数据中识别多种情绪的共现。现有方法主要关注融合策略和模态-标签依赖建模,但常忽略固有的随机不确定性(aleatoric uncertainty),即多模态数据中的固有噪声会因特征表示模糊而干扰模态融合。本文提出一种基于潜在情绪空间概率建模的新框架——带有不确定性感知的情绪分布解耦(LDDU)。具体而言,在情绪空间中引入对比解耦分布机制,以提取语义特征并建模不确定性;同时设计了一种考虑不确定性分布扩散的感知融合方法,整合分布信息。实验表明,LDDU在CMU-MOSEI和M³ED数据集上取得当前最优性能,凸显了在MMER中建模不确定性的关键作用。代码已开源:https://github.com/201983290498/lddu_mmer.git。

原文摘要 · Abstract (English)

Multimodal multi-label emotion recognition (MMER) aims to identify the concurrent presence of multiple emotions in multimodal data. Existing studies primarily focus on improving fusion strategies and modeling modality-to-label dependencies. However, they often overlook the impact of \textbf{aleatoric uncertainty}, which is the inherent noise in the multimodal data and hinders the effectiveness of modality fusion by introducing ambiguity into feature representations. To address this issue and effectively model aleatoric uncertainty, this paper proposes Latent emotional Distribution Decomposition with Uncertainty perception (LDDU) framework from a novel perspective of latent emotional space probabilistic modeling. Specifically, we introduce a contrastive disentangled distribution mechanism within the emotion space to model the multimodal data, allowing for the extraction of semantic features and uncertainty. Furthermore, we design an uncertainty-aware fusion multimodal method that accounts for the dispersed distribution of uncertainty and integrates distribution information. Experimental results show that LDDU achieves state-of-the-art performance on the CMU-MOSEI and M$^3$ED datasets, highlighting the importance of uncertainty modeling in MMER. Code is available at https://github.com/201983290498/lddu\_mmer.git.

情绪识别不确定性多模态概率建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。