用大模型分析课堂行为，不存视频也能看懂学生注意力
Can LLMs Reason About Attention? Towards Zero-Shot Analysis of Multimodal Classroom Behavior
- 用人体姿态+视线估计提取数据，视频帧即时删除
- 大模型零样本分析注意力，生成热力图与摘要报告
- 适合教育科技研究者，隐私保护下做行为分析
传统课堂注意力分析依赖耗时的人工观察或侵入式录制，存在隐私风险。本文提出一种隐私保护的分析流程：通过OpenPose提取骨骼关键点，用Gaze-LLE估算视觉注意力，原始视频帧在提取后立即删除，仅保留几何坐标（以JSON格式存储），符合FERPA要求。处理后的姿态与视线数据由QwQ-32B-Reasoning模型进行零样本行为分析，按授课片段生成注意力分布。教师可通过网页仪表盘查看注意力热力图与行为摘要。初步结果表明，大模型在多模态行为理解方面具有潜力，但仍难以准确进行教室空间布局的空间推理。论文讨论了局限性，并提出改进大模型空间认知能力的方向。
原文摘要 · Abstract (English)
Understanding student engagement usually requires time-consuming manual observation or invasive recording that raises privacy concerns. We present a privacy-preserving pipeline that analyzes classroom videos to extract insights about student attention, without storing any identifiable footage. Our system runs on a single GPU, using OpenPose for skeletal extraction and Gaze-LLE for visual attention estimation. Original video frames are deleted immediately after pose extraction, thus only geometric coordinates (stored as JSON) are retained, ensuring compliance with FERPA. The extracted pose and gaze data is processed by QwQ-32B-Reasoning, which performs zero-shot analysis of student behavior across lecture segments. Instructors access results through a web dashboard featuring attention heatmaps and behavioral summaries. Our preliminary findings suggest that LLMs may show promise for multimodal behavior understanding, although they still struggle with spatial reasoning about classroom layouts. We discuss these limitations and outline directions for improving LLM spatial comprehension in educational analytics contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。