融合多模态专家与人类偏好，提升情绪识别准确率
More Is Better: A MoE-Based Emotion Recognition Framework with Human Preference Alignment
- 用多模态信号作为独立专家，构建混合专家模型
- 通过共识伪标签策略利用未标注数据，提升模型性能
- 投票+规则重排机制对齐人类偏好，适合高精度情绪分析场景
本文针对MER2025半监督学习赛道提出一种基于'更多更好'原则的综合框架，构建稳健的混合专家（MoE）情绪识别系统。方法融合多种输入模态作为独立专家，包括来自大视觉语言模型（VLMs）的知识和时序动作单元（AU）信息等新信号。为有效利用未标注数据，引入基于共识的伪标签策略，通过基线模型与Gemini的一致性生成高质量伪标签，并采用两阶段训练范式。最后结合多专家投票与规则化重排序过程，修正预测偏差，更好地对齐人类偏好。在MER2025-SEMI挑战数据集上，测试集F1得分为0.8772，排名该赛道第二。代码已公开于https://github.com/zhuyjan/MER2025-MRAC25。
原文摘要 · Abstract (English)
In this paper, we present our solution for the semi-supervised learning track (MER-SEMI) in MER2025. We propose a comprehensive framework, grounded in the principle that "more is better," to construct a robust Mixture of Experts (MoE) emotion recognition system. Our approach integrates a diverse range of input modalities as independent experts, including novel signals such as knowledge from large Vision-Language Models (VLMs) and temporal Action Unit (AU) information. To effectively utilize unlabeled data, we introduce a consensus-based pseudo-labeling strategy, generating high-quality labels from the agreement between a baseline model and Gemini, which are then used in a two-stage training paradigm. Finally, we employ a multi-expert voting ensemble combined with a rule-based re-ranking process to correct prediction bias and better align the outputs with human preferences. Evaluated on the MER2025-SEMI challenge dataset, our method achieves an F1-score of 0.8772 on the test set, ranking 2nd in the track. Our code is available at https://github.com/zhuyjan/MER2025-MRAC25.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。