首个真实工业场景的双视角行为数据集，助力人机协作安全研究。
ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios
- 采集180组同步双视角视频，覆盖真实工业流程。
- 标注时空信息，支持动作分割、关键步骤识别等任务。
- 提供基线模型评估，推动跨视角行为理解新方法发展。
从互补的内视角（ego）和外视角（exo）理解人类行为，有助于开发支持工业工人并提升其安全性的系统。然而，该领域进展受限于缺乏在真实工业场景中同时捕获两种视角的数据集。为填补这一空白，我们提出ENIGMA-360，一个在真实工业环境中采集的新型双视角数据集。该数据集包含180段内视角视频与180段外视角视频，时间上严格同步，提供了同一场景的互补信息。全部360段视频均带有时空标注，支持对工业场景下人类行为多维度的研究。我们为三项基础任务提供基准实验：1）时间动作分割，2）关键步骤识别，3）内视角人-物交互检测，揭示了现有先进方法在此挑战性场景中的局限性。这些结果凸显了构建具备鲁棒双视角理解能力的新模型的必要性。数据集及其标注已公开发布于https://fpv-iplab.github.io/ENIGMA-360/。
原文摘要 · Abstract (English)
Understanding human behavior from complementary egocentric (ego) and exocentric (exo) points of view enables the development of systems that can support workers in industrial environments and enhance their safety. However, progress in this area is hindered by the lack of datasets capturing both views in realistic industrial scenarios. To address this gap, we propose ENIGMA-360, a new ego-exo dataset acquired in a real industrial scenario. The dataset is composed of 180 egocentric and 180 exocentric procedural videos temporally synchronized offering complementary information of the same scene. The 360 videos have been labeled with temporal and spatial annotations, enabling the study of different aspects of human behavior in industrial domain. We provide baseline experiments for 3 foundational tasks for human behavior understanding: 1) Temporal Action Segmentation, 2) Keystep Recognition and 3) Egocentric Human-Object Interaction Detection, showing the limits of state-of-the-art approaches on this challenging scenario. These results highlight the need for new models capable of robust ego-exo understanding in real-world environments. We publicly release the dataset and its annotations at https://fpv-iplab.github.io/ENIGMA-360/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。