构建了多模态分层标注的日常活动数据集，用于理解真实场景中人类行为。
Hierarchical and Multimodal Data for Daily Activity Understanding
- 采用三层分层标注：高层任务、低层动作、细粒度步骤
- 22.7%的低层动作跨高层任务共享，14.2%的细粒度步骤跨动作共享
- 支持多传感器融合与反事实分析，适合人机交互研究
DARai（Daily Activity Recordings for Artificial Intelligence）是一个多模态、分层标注的数据集，旨在理解真实环境中的人类活动。该数据集包含50名参与者在10种不同环境中的连续脚本化与非脚本化录制，总计超过200小时数据，涵盖20种传感器，包括多视角摄像头、深度与雷达传感器、可穿戴惯性测量单元（IMUs）、肌电图（EMG）、鞋垫压力传感器、生物监测传感器及视线追踪器。为捕捉人类活动的复杂性，数据集在三个层级进行标注：(i) 高层级活动（L1），即独立任务；(ii) 低层级动作（L2），在不同活动中共享的模式；(iii) 细粒度程序（L3），详细描述动作的具体执行步骤。标注设计确保22.7%的L2动作在不同L1活动间共享，14.2%的L3程序在不同L2动作间共享。数据集的重叠与非脚本特性支持反事实活动分析。通过多种机器学习模型实验，验证了其在识别、时间定位与未来动作预测等任务中的价值。同时，基于多传感器与反事实设置，开展了领域差异性实验，揭示单个传感器的局限性。代码、文档与数据集已公开于DARai官网。
原文摘要 · Abstract (English)
Daily Activity Recordings for Artificial Intelligence (DARai, pronounced "Dahr-ree") is a multimodal, hierarchically annotated dataset constructed to understand human activities in real-world settings. DARai consists of continuous scripted and unscripted recordings of 50 participants in 10 different environments, totaling over 200 hours of data from 20 sensors including multiple camera views, depth and radar sensors, wearable inertial measurement units (IMUs), electromyography (EMG), insole pressure sensors, biomonitor sensors, and gaze tracker. To capture the complexity in human activities, DARai is annotated at three levels of hierarchy: (i) high-level activities (L1) that are independent tasks, (ii) lower-level actions (L2) that are patterns shared between activities, and (iii) fine-grained procedures (L3) that detail the exact execution steps for actions. The dataset annotations and recordings are designed so that 22.7% of L2 actions are shared between L1 activities and 14.2% of L3 procedures are shared between L2 actions. The overlap and unscripted nature of DARai allows counterfactual activities in the dataset. Experiments with various machine learning models showcase the value of DARai in uncovering important challenges in human-centered applications. Specifically, we conduct unimodal and multimodal sensor fusion experiments for recognition, temporal localization, and future action anticipation across all hierarchical annotation levels. To highlight the limitations of individual sensors, we also conduct domain-variant experiments that are enabled by DARai's multi-sensor and counterfactual activity design setup. The code, documentation, and dataset are available at the dedicated DARai website: https://alregib.ece.gatech.edu/software-and-datasets/darai-daily-activity-recordings-for-artificial-intelligence-and-machine-learning/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。