动态对齐多模态情感信号，提升在线教育情绪识别的鲁棒性。
Edu-EmotionNet: Cross-Modality Attention Alignment with Temporal Feedback Loops
- 通过跨模态注意力动态共享信息，实现多源数据融合。
- 在IEMOCAP和MOSEI上达到当前最优性能，抗噪声与缺失模态能力强。
- 适合实时在线教学系统，可自适应判断各信号可靠性。
在线教育中理解学习者情绪对提升参与度与个性化教学至关重要。现有情绪识别方法虽探索了多模态融合与时间建模，但通常依赖静态融合策略，且假设各模态输入始终可靠，这在真实学习环境中并不成立。本文提出Edu-EmotionNet，一种联合建模情绪时序演化与模态可靠性的新框架。包含三个核心组件：跨模态注意力对齐（CMAA）模块，用于动态跨模态上下文共享；模态重要性估计器（MIE），在每个时间步为各模态分配基于置信度的权重；时间反馈回路（TFL），利用历史预测强化时序一致性。在IEMOCAP和MOSEI的教育子集上评估，数据经重新标注以包含困惑、好奇、无聊与挫败四种情绪。结果表明，Edu-EmotionNet达到当前最优表现，并展现出对缺失或噪声模态的强大鲁棒性。可视化证实其能捕捉情绪转变过程并自适应优先处理可靠信号，适用于实时学习系统部署。
原文摘要 · Abstract (English)
Understanding learner emotions in online education is critical for improving engagement and personalized instruction. While prior work in emotion recognition has explored multimodal fusion and temporal modeling, existing methods often rely on static fusion strategies and assume that modality inputs are consistently reliable, which is rarely the case in real-world learning environments. We introduce Edu-EmotionNet, a novel framework that jointly models temporal emotion evolution and modality reliability for robust affect recognition. Our model incorporates three key components: a Cross-Modality Attention Alignment (CMAA) module for dynamic cross-modal context sharing, a Modality Importance Estimator (MIE) that assigns confidence-based weights to each modality at every time step, and a Temporal Feedback Loop (TFL) that leverages previous predictions to enforce temporal consistency. Evaluated on educational subsets of IEMOCAP and MOSEI, re-annotated for confusion, curiosity, boredom, and frustration, Edu-EmotionNet achieves state-of-the-art performance and demonstrates strong robustness to missing or noisy modalities. Visualizations confirm its ability to capture emotional transitions and adaptively prioritize reliable signals, making it well suited for deployment in real-time learning systems
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。