用双流结构融合动作与场景信息,提升课堂小组参与度识别准确率。
Attention-Guided Dual-Stream Learning for Group Engagement Recognition: Fusing Transformer-Encoded Motion Dynamics with Scene Context via Adaptive Gating
- 双流设计:一轨追踪个体动作,一轨捕捉整体场景动态
- 在自建数据集上达96.21%准确率,F1值0.9530
- 自适应门控融合机制,适合教育智能监控场景
学生参与度对小组学习成效至关重要。现有自动化方法多针对在线课堂或个体层面,难以捕捉小组协同行为。本文提出DualEngage,一种基于双流架构的课堂小组参与度识别新方法,联合建模个体与群体行为。主流通过检测与追踪学生,使用RAPF网络提取密集光流,经Transformer编码时序运动模式,并通过注意力池化聚合为统一表征;辅流则利用预训练3D ResNet捕获视频全局时空信息。两路特征通过软最大门控融合,动态调整贡献权重。在青岛海洋大学自建的Classroom Group Engagement Dataset上,五折交叉验证平均准确率达0.9621±0.0161,宏平均F1为0.9530±0.0204。消融实验验证了双流结构的有效性,本工作是首个显式引入运动线索的课堂参与度双流研究。
原文摘要 · Abstract (English)
Student engagement is crucial for improving learning outcomes in group activities. Highly engaged students perform better both individually and contribute to overall group success. However, most existing automated engagement recognition methods are designed for online classrooms or estimate engagement at the individual level. Addressing this gap, we propose DualEngage, a novel two-stream framework for group-level engagement recognition from in-classroom videos. It models engagement as a joint function of both individual and group-level behaviors. The primary stream models person-level motion dynamics by detecting and tracking students, extracting dense optical flow with the Recurrent All-Pairs Field Transforms network, encoding temporal motion patterns using a transformer encoder, and finally aggregating per-student representations through attention pooling into a unified representation. The secondary stream captures scene-level spatiotemporal information from the full video clip, leveraging a pretrained three-dimensional Residual Network. The two-stream representations are combined via softmax-gated fusion, which dynamically weights each stream's contribution based on the joint context of both features. DualEngage learns a joint representation of individual actions with overarching group dynamics. We evaluate the proposed approach using fivefold cross-validation on the Classroom Group Engagement Dataset developed by Ocean University of China, achieving an average classification accuracy of 0.9621+/-0.0161 with a macro-averaged F1 of 0.9530+/-0.0204. To understand the contribution of each branch, we further conduct an ablation study comparing single-stream variants against the two-stream model. This work is among the first in classroom engagement recognition to adopt a dual-stream design that explicitly leverages motion cues as an estimator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。