用事件相机实现低延迟高精度眼动追踪,专为扩展现实设计。
FACET: Fast and Accurate Event-Based Eye Tracking Using Ellipse Modeling for Extended Reality
- 直接从事件数据输出瞳孔椭圆参数,端到端实时处理。
- 误差仅0.20像素,推理时间0.53毫秒,性能超越前代1.6倍以上。
- 适合需要低功耗、高帧率的眼动交互系统,如XR头显。
眼动追踪是扩展现实(XR)中基于视线交互的关键技术,但传统帧基系统难以满足高精度、低延迟和低功耗的需求。事件相机凭借高时间分辨率和低功耗成为有前景的替代方案。本文提出FACET(Fast and Accurate Event-based Eye Tracking),一个端到端神经网络,可直接从事件数据输出瞳孔椭圆参数,专为实时XR应用优化。通过扩充并转换EV-Eye数据集的标注为椭圆形式以训练模型,并引入新型三角损失解决角度不连续问题,同时提出一种快速因果事件体积表示方法。在增强后的EV-Eye测试集上,FACET平均瞳孔中心误差达0.20像素,推理时间仅0.53毫秒,相比先前方法EV-Eye分别降低1.6倍误差和1.8倍延迟,参数量与运算量分别减少4.4倍和11.7倍。代码已公开于https://github.com/DeanJY/FACET。
原文摘要 · Abstract (English)
Eye tracking is a key technology for gaze-based interactions in Extended Reality (XR), but traditional frame-based systems struggle to meet XR's demands for high accuracy, low latency, and power efficiency. Event cameras offer a promising alternative due to their high temporal resolution and low power consumption. In this paper, we present FACET (Fast and Accurate Event-based Eye Tracking), an end-to-end neural network that directly outputs pupil ellipse parameters from event data, optimized for real-time XR applications. The ellipse output can be directly used in subsequent ellipse-based pupil trackers. We enhance the EV-Eye dataset by expanding annotated data and converting original mask labels to ellipse-based annotations to train the model. Besides, a novel trigonometric loss is adopted to address angle discontinuities and a fast causal event volume event representation method is put forward. On the enhanced EV-Eye test set, FACET achieves an average pupil center error of 0.20 pixels and an inference time of 0.53 ms, reducing pixel error and inference time by 1.6$\times$ and 1.8$\times$ compared to the prior art, EV-Eye, with 4.4$\times$ and 11.7$\times$ less parameters and arithmetic operations. The code is available at https://github.com/DeanJY/FACET.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。