用事件相机+时序建模,实现高速高精度眼动追踪
Exploring Temporal Dynamics in Event-based Eye Tracker
- 从隐式和显式两方面建模时间动态,提升眼动捕捉能力
- 在合成数据集上达到顶尖性能,斩获CVPR挑战赛第三名
- 适合关注低功耗、高帧率眼动追踪的AR/VR开发者
眼动追踪是人机交互关键技术,尤其在增强现实(AR)、虚拟现实(VR)及扩展现实(XR)可穿戴设备中至关重要。基于帧的图像传感器因时间分辨率有限,难以准确捕捉快速眼动(如扫视和眨眼)等瞬态动态。事件相机模仿生物视觉系统,具备极低功耗与超高的时间分辨率,是实现高速高精度眼动追踪的理想方案。本文提出TDTracker框架,通过隐式与显式双重方式建模时间动态:利用3D卷积网络捕捉短时隐式动态,并采用包含频率感知模块、GRU与Mamba的级联结构提取长时显式动态;最终通过预测热图完成眼坐标回归。实验表明,TDTracker在合成数据集SEET上达到当前最优(SOTA)表现,并在CVPR 2025事件相机眼动追踪挑战赛中荣获第三名。代码已开源。
原文摘要 · Abstract (English)
Eye-tracking is a vital technology for human-computer interaction, especially in wearable devices such as AR, VR, and XR. The realization of high-speed and high-precision eye-tracking using frame-based image sensors is constrained by their limited temporal resolution, which impairs the accurate capture of rapid ocular dynamics, such as saccades and blinks. Event cameras, inspired by biological vision systems, are capable of perceiving eye movements with extremely low power consumption and ultra-high temporal resolution. This makes them a promising solution for achieving high-speed, high-precision tracking with rich temporal dynamics. In this paper, we propose TDTracker, an effective eye-tracking framework that captures rapid eye movements by thoroughly modeling temporal dynamics from both implicit and explicit perspectives. TDTracker utilizes 3D convolutional neural networks to capture implicit short-term temporal dynamics and employs a cascaded structure consisting of a Frequency-aware Module, GRU, and Mamba to extract explicit long-term temporal dynamics. Ultimately, a prediction heatmap is used for eye coordinate regression. Experimental results demonstrate that TDTracker achieves state-of-the-art (SOTA) performance on the synthetic SEET dataset and secured Third place in the CVPR event-based eye-tracking challenge 2025. Our code is available at https://github.com/rhwxmx/TDTracker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。