提出首个360度视角下的人类注视目标估计系统，提升机器人感知能力。
GazeTarget360: Towards Gaze Target Estimation in 360-Degree for Robot Perception
- 融合眼接触检测器、视觉编码器与多尺度解码器实现360度注视预测
- 在未见过场景中仍能准确预测注视目标，跨场景泛化能力强
- 适用于真实机器人交互场景，代码开源可直接部署
让机器人理解人类注视目标是实现注意力估计与行为预判等下游任务的关键。以往方法依赖数据驱动，在图像内定位注视目标时需剔除图像外样本，而基于视觉的注视估计（如OpenFace）难以利用背景信息，无法处理视线偏离摄像头的情况。本文提出GazeTarget360系统，用于在一般视觉场景中从图像实现360度注视目标估计。该系统集成眼接触检测器、预训练视觉编码器与多尺度融合解码器，交叉验证结果表明其可在未见过的场景中生成准确可靠的注视目标预测，是首个能从真实摄像机画面高效预测360度注视目标的系统。源代码已公开于：https://github.com/zdai257/DisengageNet。
原文摘要 · Abstract (English)
Enabling robots to understand human gaze target is a crucial step to allow capabilities in downstream tasks, for example, attention estimation and movement anticipation in real-world human-robot interactions. Prior works have addressed the in-frame target localization problem with data-driven approaches by carefully removing out-of-frame samples. Vision-based gaze estimation methods, such as OpenFace, do not effectively absorb background information in images and cannot predict gaze target in situations where subjects look away from the camera. In this work, we propose a system to address the problem of 360-degree gaze target estimation from an image in generalized visual scenes. The system, named GazeTarget360, integrates conditional inference engines of an eye-contact detector, a pre-trained vision encoder, and a multi-scale-fusion decoder. Cross validation results show that GazeTarget360 can produce accurate and reliable gaze target predictions in unseen scenarios. This makes a first-of-its-kind system to predict gaze targets from realistic camera footage which is highly efficient and deployable. Our source code is made publicly available at: https://github.com/zdai257/DisengageNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。