无需标注数据，用新模型实现高精度虚拟现实眼球追踪。
GazeShift: Unsupervised Gaze Estimation and Dataset for VR
- 提出注意力引导的无监督框架，从近眼红外图像中学习眼球特征。
- 在210万张图像上训练，用户校准后误差仅1.84°，远超基线。
- 模型轻量实时，适配头显部署，适合做低资源眼球追踪系统。
眼球追踪对现代虚拟现实系统至关重要。尽管远程摄像头眼球追踪取得进展，但VR研究受限于数据稀缺，尤其缺乏大规模、精准标注的离轴摄像头配置数据集。由于难以确保注视目标，标注极为困难。为此，我们推出VRGaze——首个面向VR的大型离轴眼球追踪数据集，包含68名参与者采集的210万张近眼红外图像。我们进一步提出GazeShift，一种基于注意力引导的无监督框架，可在无标注数据下学习眼球表征。与依赖多视角或3D几何的现有方法不同，GazeShift专为近眼图像设计，实现了高效的眼球-外观解耦，且模型紧凑、实时性强。通过轻量少样本校准，可适配个体用户，在VRGaze上达到1.84°均方误差；在远程摄像头MPIIGaze数据集上，实现7.15°无特定人误差，参数量减少10倍，浮点运算量减少35倍。部署于头显GPU时，推理仅需5毫秒。结合对光照变化的鲁棒性，表明GazeShift是标签高效、实时的VR眼球追踪解决方案。项目代码与数据集已开源。
原文摘要 · Abstract (English)
Gaze estimation is instrumental in modern virtual reality (VR) systems. Despite significant progress in remote-camera gaze estimation, VR gaze research remains constrained by data scarcity, particularly the lack of large-scale, accurately labeled datasets captured with the off-axis camera configurations typical of modern headsets. Gaze annotation is difficult since fixation on intended targets cannot be guaranteed. To address these challenges, we introduce VRGaze, the first large-scale off-axis gaze estimation dataset for VR, comprising 2.1 million near-eye infrared images collected from 68 participants. We further propose GazeShift, an attention-guided unsupervised framework for learning gaze representations without labeled data. Unlike prior redirection-based methods that rely on multi-view or 3D geometry, GazeShift is tailored to near-eye imagery, achieving effective gaze-appearance disentanglement in a compact, real-time model. GazeShift embeddings can be optionally adapted to individual users via lightweight few-shot calibration, achieving a 1.84° mean error on VRGaze. On the remote-camera MPIIGaze dataset, the model achieves a 7.15° person-agnostic error, doing so with 10x fewer parameters and 35x fewer FLOPs than baseline methods. Deployed natively on a VR headset GPU, inference takes only 5 ms. Combined with demonstrated robustness to illumination changes, these results highlight GazeShift as a label-efficient, real-time solution for VR gaze tracking. Project code and the VRGaze dataset are released at https://github.com/gazeshift3/gazeshift
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。