用脉冲神经网络实现低功耗高实时的眼动追踪,精度接近专用系统。
Neuromorphic Eye Tracking for Low-Latency Pupil Detection
- 将事件相机眼动模型转为脉冲神经网络,用轻量LIF层替代复杂模块
- 误差仅3.7-4.1像素,模型大小缩小20倍,计算量减少850倍
- 适合AR/VR等可穿戴设备,实测功耗仅3.9-4.9毫瓦,延迟3毫秒
可穿戴系统中的眼动追踪需低延迟与毫瓦级功耗,但传统帧基流程易受运动模糊影响,计算开销大且时间分辨率低。这一能力对增强现实(AR)与虚拟现实(VR)中实现自然响应的交互至关重要。脉冲神经网络(SNN)与类脑传感器提供了新路径,但现有SNN方法或过于专用,或性能不及先进ANN架构。本文提出一种面向事件的眼动追踪模型的类脑版本,将原模型中的循环与注意力模块替换为轻量级积分发放(LIF)层,并采用深度可分离卷积降低模型复杂度。所提模型均取得3.7-4.1像素的平均误差,接近专用类脑系统Retina(3.24像素)的精度;相比最接近的ANN变体,模型规模缩小20倍,理论计算量减少850倍。预计在1kHz采样率下,运行功耗仅为3.9-4.9毫瓦,延迟3毫秒。结果表明,高性能事件驱动眼动追踪架构可高效重构为SNN,在保持实时可穿戴部署精度的同时获得显著能效优势。
原文摘要 · Abstract (English)
Eye tracking for wearable systems demands low latency and milliwatt-level power, but conventional frame-based pipelines struggle with motion blur, high compute cost, and limited temporal resolution. Such capabilities are vital for enabling seamless and responsive interaction in emerging technologies like augmented reality (AR) and virtual reality (VR), where understanding user gaze is key to immersion and interface design. Neuromorphic sensors and spiking neural networks (SNNs) offer a promising alternative, yet existing SNN approaches are either too specialized or fall short of the performance of modern ANN architectures. This paper presents a neuromorphic version of top-performing event-based eye-tracking models, replacing their recurrent and attention modules with lightweight LIF layers and exploiting depth-wise separable convolutions to reduce model complexity. Our models obtain 3.7-4.1px mean error, approaching the accuracy of the application-specific neuromorphic system, Retina (3.24px), while reducing model size by 20x and theoretical compute by 850x, compared to the closest ANN variant of the proposed model. These efficient variants are projected to operate at an estimated 3.9-4.9 mW with 3 ms latency at 1 kHz. The present results indicate that high-performing event-based eye-tracking architectures can be redesigned as SNNs with substantial efficiency gains, while retaining accuracy suitable for real-time wearable deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。