构建工业场景下人机共存动作数据集，助力机器人安全交互
THÖR-MAGNI Act: Actions for Human Motion Modeling in Robot-Shared Industrial Spaces
- 基于眼动眼镜采集8.3小时带精细动作标签的工业场景视频
- 提出两个高效Transformer模型，在动作与轨迹预测任务上优于基线
- 适用于需精准预测人类行为的工业机器人协作场景
准确预测动态环境中人员活动与轨迹对保障人机交互安全至关重要，尤其在配备移动机器人的工业场景中。现有数据集多聚焦公共空间的社会导航，缺乏工业环境下带细粒度动作标签的数据。本文推出THÖR-MAGNI Act数据集，是THÖR-MAGNI数据集的重大扩展，记录了参与者在不同语义与空间背景下与机器人共处的运动行为。该数据集包含8.3小时经人工标注的动作标签，来源于佩戴眼动仪采集的视角视频。动作标签与原始的THÖR-MAGNI运动线索对齐，具有多样化的加速度、速度和导航距离分布，符合长尾特征。我们展示了该数据集在两项任务中的应用价值：动作条件下的轨迹预测与联合动作与轨迹预测。为此提出了两种高效的Transformer模型，性能显著优于基线方法。结果表明，THÖR-MAGNI Act具备推动复杂环境下人机预测模型发展的潜力。
原文摘要 · Abstract (English)
Accurate human activity and trajectory prediction are crucial for ensuring safe and reliable human-robot interactions in dynamic environments, such as industrial settings, with mobile robots. Datasets with fine-grained action labels for moving people in industrial environments with mobile robots are scarce, as most existing datasets focus on social navigation in public spaces. This paper introduces the THÖR-MAGNI Act dataset, a substantial extension of the THÖR-MAGNI dataset, which captures participant movements alongside robots in diverse semantic and spatial contexts. THÖR-MAGNI Act provides 8.3 hours of manually labeled participant actions derived from egocentric videos recorded via eye-tracking glasses. These actions, aligned with the provided THÖR-MAGNI motion cues, follow a long-tailed distribution with diversified acceleration, velocity, and navigation distance profiles. We demonstrate the utility of THÖR-MAGNI Act for two tasks: action-conditioned trajectory prediction and joint action and trajectory prediction. We propose two efficient transformer-based models that outperform the baselines to address these tasks. These results underscore the potential of THÖR-MAGNI Act to develop predictive models for enhanced human-robot interaction in complex environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。