arXiv:2608.07867cs.LG2026-08

提出冲突感知框架,让多模态情绪识别自动校准不可靠标签。

CONFER: Conflict-Aware Evidence Negotiation for Regime-Calibrated Weak Supervision in Multimodal Emotion Recognition

论文配图:CONFER: Conflict-Aware Evidence Negotiation for Regime-Calibrated Weak Supervision in Multimodal Emotion Recognition
图 1 · 摘自论文原文
  • 用图结构建模各模态专家,通过不确定性与可靠性动态协商。
  • 在严格留一被试测试中达到0.873和0.854的准确率。
  • 适合处理标签不可靠、模态冲突明显的多模态情绪识别任务。

多模态情绪识别常将自报告标签视为可靠监督,却忽视其不可靠性与跨模态冲突。本文提出基于图结构的冲突感知证据协商框架CONFER,将每个模态专家表示为节点,包含预测信念、基于边界的不确定性及从历史留出样本表现与当前样本不确定性估计的运行时可靠性。不确定性感知的兼容性与可靠性导向的非对称边权重驱动迭代消息传递协商,随后进行同伴支持的预测读出。冲突减少、残差分歧与平均模态不确定性共同刻画了三种样本特定的模式——共识、分歧与模糊,用于弱标签校准。在AMIGOS、MAHNOB-HCI和DEAP数据集上,采用被试内10折与严格留一被试外(LOSO)协议评估,CONFER表现优异,在严格LOSO下于AMIGOS-V达0.873准确率,MAHNOB-V达0.854。进一步分析显示,高冲突样本获得更大协商增益,且对弱标签污染更具鲁棒性,表明跨模态冲突可为模态协调与监督可靠性估计提供有效信息。

原文摘要 · Abstract (English)

Multimodal emotion recognition often treats self-reported labels as reliable supervision while overlooking self-report unreliability and cross-modal conflict. We propose \textbf{CONFER}, a graph-based conflict-aware evidence negotiation framework for weakly supervised multimodal emotion recognition. CONFER represents each modality expert as a node with a predictive belief, boundary-based uncertainty, and runtime reliability estimated from historical out-of-fold performance and current-sample uncertainty. Uncertainty-aware compatibility and reliability-directed asymmetric edge weights govern iterative message-passing negotiation, followed by peer-supported prediction readout. Conflict reduction, residual disagreement, and mean modality uncertainty further characterize three regimes---Consensus, Dissent, and Ambiguity---for sample-specific weak-label calibration. We evaluate CONFER on AMIGOS, MAHNOB-HCI, and DEAP under subject-dependent 10-fold and strict leave-one-subject-out (LOSO) protocols. CONFER achieves competitive performance, reaching \textbf{0.873} accuracy on AMIGOS-V and \textbf{0.854} accuracy on MAHNOB-V under strict LOSO evaluation. Further analyses show larger negotiation gains on high-conflict samples and improved robustness to weak-label corruption, indicating that cross-modal conflict provides useful information for both directional modality coordination and supervision-reliability estimation.

多模态情绪识别弱监督冲突感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。