用3D合成数据模拟眼动追踪硬件性能,加速原型设计。
Digitally Prototype Your Eye Tracker: Simulating Hardware Performance using 3D Synthetic Data
- 基于真实3D眼球重建,生成多视角合成数据。
- 可预测不同硬件配置下的眼动追踪相对性能。
- 无需实物即可评估相机位置对性能的影响。
眼动追踪(ET)是增强现实与虚拟现实(AR/VR)的关键技术。新硬件的原型设计需评估硬件选择对性能的影响,但真实硬件数据获取成本高,尤其对机器学习所需大规模训练数据而言更为困难。本文提出一种仅使用合成数据的端到端方法,评估硬件变化对基于机器学习的ET性能的影响。利用从光穹数据重建的真实3D眼球数据,通过神经辐射场(NeRF)生成新视角与相机参数下的合成眼图像。该框架可预测不同硬件配置下的性能表现,涵盖传感器噪声、光照亮度和光学模糊等变量。与公开的Project Aria眼镜眼动追踪数据集对比,模拟结果与真实性能高度相关。此外,首次系统分析了从正视到周边视角的相机位置变化对性能的影响,此类研究以往需制造实体设备采集数据。本方法显著加速了眼动追踪硬件的原型开发。
原文摘要 · Abstract (English)
Eye tracking (ET) is a key enabler for Augmented and Virtual Reality (AR/VR). Prototyping new ET hardware requires assessing the impact of hardware choices on eye tracking performance. This task is compounded by the high cost of obtaining data from sufficiently many variations of real hardware, especially for machine learning, which requires large training datasets. We propose a method for end-to-end evaluation of how hardware changes impact machine learning-based ET performance using only synthetic data. We utilize a dataset of real 3D eyes, reconstructed from light dome data using neural radiance fields (NeRF), to synthesize captured eyes from novel viewpoints and camera parameters. Using this framework, we demonstrate that we can predict the relative performance across various hardware configurations, accounting for variations in sensor noise, illumination brightness, and optical blur. We also compare our simulator with the publicly available eye tracking dataset from the Project Aria glasses, demonstrating a strong correlation with real-world performance. Finally, we present a first-of-its-kind analysis in which we vary ET camera positions, evaluating ET performance ranging from on-axis direct views of the eye to peripheral views on the frame. Such an analysis would have previously required manufacturing physical devices to capture evaluation data. In short, our method enables faster prototyping of ET hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。