arXiv:2608.11051cs.CV2026-08

构建首个360°人机交互预测数据集,支持机器人预判人类行为。

HUI360: A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation

论文配图:HUI360: A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation
图 1 · 摘自论文原文
  • 基于移动机器人在真实环境采集多日数据,覆盖多样人群与场景。
  • 发布100万条高质量标注,含2D姿态、面部关键点与分割掩码。
  • 提供跨数据集评估基准,助力模型泛化能力研究。

随着机器人越来越多地在人类环境中运行,预判人类意图对于实现主动且社交敏感的行为至关重要。自动预判人机交互正成为具身智能体的关键感知挑战。为此,我们提出HUI360,这是目前最大的野外环境下人机交互预判数据集及其基线方案。数据由移动机器人在3个月内多个不同环境中采集,涵盖自然、自发的人类行为,包含多样化个体,有助于评估和提升交互预判模型的泛化能力。我们设计了一套流程,可对任意360°等距投影视频进行自动交互标注,并提供人工修正界面。通过该流程,我们发布了100万条预处理标注数据,包括使用先进计算机视觉方法获得的详细2D姿态、面部关键点及分割掩码,并经人工校验确保高质量跟踪与交互标注。此外,我们还按需发布原始全景360°图像(仅限科研用途,符合GDPR)。最后,我们建立了交互预判基准,首次实现跨数据集评估:为此,我们也发布了另一个现有野外户外数据集SSUP-HRI的600万条标注。数据与代码见https://hucebot.github.io/hui360。

原文摘要 · Abstract (English)

As robots increasingly operate in human-populated environments, anticipating human intentions is essential for enabling proactive and socially aware behavior. Automatic anticipation of human-robot interactions is thus emerging as a crucial perception challenge for embodied agents. To this end, we introduce HUI360, the largest dataset for human-robot interaction anticipation in the wild and its set of baselines. The dataset was collected from a mobile robot, in the wild, over multiple days within a 3-month period, and in several environments, capturing natural, spontaneous behaviors from both passersby and users, and encompassing a diverse range of individuals. This variety enables evaluating and improving the generalization capabilities of interaction anticipation models. We designed a pipeline and share code for automatic interaction annotation in arbitrary 360-degree equirectangular videos, along with interfaces for manual refinement. Using this pipeline, we release the HUI360 open set of 1M pre-processed annotations, including detailed 2D poses, facial keypoints, and segmentation masks, obtained using state-of-the-art computer vision methods and manually curated to ensure high-quality tracking and interaction annotation. Additionally, we release the raw panoptic 360-degree images captured from the robot's egocentric viewpoint (on demand, for research purpose only in compliance with GDPR). Finally, we establish benchmark baselines for interaction anticipation, including the first cross-dataset evaluations for this task: to this end, we also release 6M annotations for another existing in-the-wild outdoor dataset collected from a mobile robot (SSUP-HRI). Dataset and code can be found at https://hucebot.github.io/hui360.

人机交互360度数据行为预测机器人感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。