用人脸和场景信息联合判断在线学习时的专注度,准确率达73.43%。
ViBED-Net: Video Based Engagement Detection Network Using Face-Aware and Scene-Aware Spatiotemporal Cues
- 双流结构分别处理人脸和全场景视频帧,提取时空特征
- 在DAiSEE数据集上达到73.43%准确率,优于现有方法
- 适合教育评估、个性化教学与用户体验研究
在线学习中的参与度检测对提升学生表现和个性化教学至关重要。本文提出ViBED-Net(基于视频的参与度检测网络),一种新型深度学习框架,通过双流架构从视频数据中评估学生参与度。该模型同时处理面部区域与完整画面帧,利用EfficientNetV2提取空间特征,并采用LSTM与Transformer编码器进行时间建模。在DAiSEE大规模情感状态识别基准数据集上进行评估,针对低频参与类别的性能,引入针对性数据增强策略。在多个变体中,使用LSTM的ViBED-Net达到73.43%的准确率,优于现有最优方法。结果表明,融合人脸感知与场景感知的时空线索能显著提升检测精度。其模块化设计便于应用于教育、用户体验研究及内容个性化领域。本工作推动了基于视频的情感计算发展,提供了一种可扩展、高性能的真实世界参与度分析方案。项目源码可在https://github.com/prateek-gothwal/ViBED-Net 获取。
原文摘要 · Abstract (English)
Engagement detection in online learning environments is vital for improving student outcomes and personalizing instruction. We present ViBED-Net (Video-Based Engagement Detection Network), a novel deep learning framework designed to assess student engagement from video data using a dual-stream architecture. ViBED-Net captures both facial expressions and full-scene context by processing facial crops and entire video frames through EfficientNetV2 for spatial feature extraction. These features are then analyzed over time using two temporal modeling strategies: Long Short-Term Memory (LSTM) networks and Transformer encoders. Our model is evaluated on the DAiSEE dataset, a large-scale benchmark for affective state recognition in e-learning. To enhance performance on underrepresented engagement classes, we apply targeted data augmentation techniques. Among the tested variants, ViBED-Net with LSTM achieves 73.43\% accuracy, outperforming existing state-of-the-art approaches. ViBED-Net demonstrates that combining face-aware and scene-aware spatiotemporal cues significantly improves engagement detection accuracy. Its modular design allows flexibility for application across education, user experience research, and content personalization. This work advances video-based affective computing by offering a scalable, high-performing solution for real-world engagement analysis. The source code for this project is available on https://github.com/prateek-gothwal/ViBED-Net .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。