arXiv:2510.01362cs.CV2025-10被引 5

构建首个动态追踪学习中挣扎变化的视频数据集,助力智能助教系统。

EvoStruggle: A Dataset Capturing the Evolution of Struggle across Activities and Skill Levels

  • 采集76人完成18项任务的61小时视频,标注5385段挣扎片段。
  • 模型在未见任务上达到34.56% mAP,跨活动仍具可迁移性。
  • 适合研究人机交互、自适应学习系统的开发者使用。

判断学习过程中个体何时陷入困难对优化人类学习与开发有效辅助系统至关重要。随着技能提升,挣扎类型与频率会发生变化,理解这一演变过程有助于识别学习阶段。然而,现有操作数据集未关注挣扎随时间的演变。本文构建了一个挣扎判定数据集,包含61.68小时视频、2,793段视频和5,385个标注的时间段挣扎片段,来自76名参与者。数据涵盖4类活动:打结、折纸、拼图、洗牌,共18项任务,每位参与者重复执行任务五次以捕捉技能演变。将挣扎判定定义为时间动作定位任务,旨在精确识别挣扎起止时间。实验表明,时间动作定位模型能成功学习挣扎线索,即使在未见任务或活动中也表现良好。模型在跨任务场景下平均mAP达34.56%,跨活动为19.24%,说明挣扎具有跨任务可迁移性,但仍面临挑战。数据集已开源:https://github.com/FELIXFENG2019/EvoStruggle。

原文摘要 · Abstract (English)

The ability to determine when a person struggles during skill acquisition is crucial for both optimizing human learning and enabling the development of effective assistive systems. As skills develop, the type and frequency of struggles tend to change, and understanding this evolution is key to determining the user's current stage of learning. However, existing manipulation datasets have not focused on how struggle evolves over time. In this work, we collect a dataset for struggle determination, featuring 61.68 hours of video recordings, 2,793 videos, and 5,385 annotated temporal struggle segments collected from 76 participants. The dataset includes 18 tasks grouped into four diverse activities -- tying knots, origami, tangram puzzles, and shuffling cards, representing different task variations. In addition, participants repeated the same task five times to capture their evolution of skill. We define the struggle determination problem as a temporal action localization task, focusing on identifying and precisely localizing struggle segments with start and end times. Experimental results show that Temporal Action Localization models can successfully learn to detect struggle cues, even when evaluated on unseen tasks or activities. The models attain an overall average mAP of 34.56% when generalizing across tasks and 19.24% across activities, indicating that struggle is a transferable concept across various skill-based tasks while still posing challenges for further improvement in struggle detection. Our dataset is available at https://github.com/FELIXFENG2019/EvoStruggle.

行为分析学习评估视频理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。