构建细粒度石器制作动作数据集,挑战复杂手部协同行为识别。
Human Stone Toolmaking Action Grammar (HSTAG): A Challenging Benchmark for Fine-grained Motor Behavior Recognition
- 基于专家石器制作视频构建细粒度动作数据集
- 18,739段视频涵盖4.5小时专家操作,动作短暂且频繁切换
- 适合研究复杂手部交互与多视角识别的AI模型
过去十年间,动作识别领域涌现了大量新算法和数据集。然而,多数公开基准聚焦日常生活活动,标注层级粗略,缺乏特定领域的多样性,尤其在罕见领域表现不足。本文提出人类石器制作动作语法(HSTAG),一个精心标注的视频数据集,记录了此前未被记载的石器制作行为,可用于探索先进人工智能技术在理解两手持工具快速连续交互中的应用。HSTAG包含18,739个视频片段,总时长4.5小时,展现专家在石器制作中的实际操作。其独特之处在于:(i) 动作持续时间短、转换频繁,反映许多运动行为的本质快速变化;(ii) 多角度拍摄及多工具切换,提升类内差异性;(iii) 类别分布不均,不同动作序列高度相似,增加模式区分难度。使用主流动作识别模型进行实验分析,验证了HSTAG的挑战性与独特性。数据集可访问:https://nyu.databrary.org/volume/1697。
原文摘要 · Abstract (English)
Action recognition has witnessed the development of a growing number of novel algorithms and datasets in the past decade. However, the majority of public benchmarks were constructed around activities of daily living and annotated at a rather coarse-grained level, which lacks diversity in domain-specific datasets, especially for rarely seen domains. In this paper, we introduced Human Stone Toolmaking Action Grammar (HSTAG), a meticulously annotated video dataset showcasing previously undocumented stone toolmaking behaviors, which can be used for investigating the applications of advanced artificial intelligence techniques in understanding a rapid succession of complex interactions between two hand-held objects. HSTAG consists of 18,739 video clips that record 4.5 hours of experts' activities in stone toolmaking. Its unique features include (i) brief action durations and frequent transitions, mirroring the rapid changes inherent in many motor behaviors; (ii) multiple angles of view and switches among multiple tools, increasing intra-class variability; (iii) unbalanced class distributions and high similarity among different action sequences, adding difficulty in capturing distinct patterns for each action. Several mainstream action recognition models are used to conduct experimental analysis, which showcases the challenges and uniqueness of HSTAG https://nyu.databrary.org/volume/1697.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。