arXiv:2510.01213eess.SPcs.AR2025-10中稿 · 2026 IEEE 31st Asi…

12nm芯片实现每帧18.9微焦耳的超低功耗眼动追踪,支持2000帧/秒实时响应。

JaneEye: A 12-nm 2K-FPS 18.9-$μ$J/Frame Event-based Eye Tracking Accelerator

论文配图:JaneEye: A 12-nm 2K-FPS 18.9-$μ$J/Frame Event-based Eye Tracking Accelerator
图 1 · 摘自论文原文
  • 用事件数据+轻量神经网络,只用17.6K参数实现高精度眼动追踪。
  • 在3ET+数据集上误差仅2.45像素,支持最高1250赫兹事件帧率。
  • 软硬件协同设计,12纳米芯片实现0.5毫秒端到端延迟,适合下一代头戴设备。

眼动追踪是扩展现实(XR)中基于视线交互的关键技术。传统帧式系统难以满足XR对高精度、低延迟和低功耗的严苛要求。事件相机提供高时间分辨率与低功耗的替代方案。本文提出JaneEye,一款专为可穿戴设备设计的能量高效事件基眼动追踪硬件加速器,利用稀疏的高时间分辨率事件数据。我们设计了一种超轻量神经网络架构,包含新型ConvJANET层,通过仅保留遗忘门简化传统ConvLSTM,计算复杂度减半且不损失时序建模能力。模型在3ET+数据集上达到2.45像素误差,仅需17.6K参数,支持高达1250 Hz的事件帧率。为提升硬件效率,采用自定义激活函数线性近似(hardsigmoid、hardtanh)和定点量化。通过软硬件协同设计,12纳米ASIC实现400 MHz工作频率,端到端延迟仅0.5毫秒(等效2000帧/秒),能效达18.9 μJ/帧。JaneEye为下一代可穿戴XR设备提供了低功耗、高性能的眼动追踪新标准。

原文摘要 · Abstract (English)

Eye tracking has become a key technology for gaze-based interactions in Extended Reality (XR). However, conventional frame-based eye-tracking systems often fall short of XR's stringent requirements for high accuracy, low latency, and energy efficiency. Event cameras present a compelling alternative, offering ultra-high temporal resolution and low power consumption. In this paper, we present JaneEye, an energy-efficient event-based eye-tracking hardware accelerator designed specifically for wearable devices, leveraging sparse, high-temporal-resolution event data. We introduce an ultra-lightweight neural network architecture featuring a novel ConvJANET layer, which simplifies the traditional ConvLSTM by retaining only the forget gate, thereby halving computational complexity without sacrificing temporal modeling capability. Our proposed model achieves high accuracy with a pixel error of 2.45 on the 3ET+ dataset, using only 17.6K parameters, with up to 1250 Hz event frame rate. To further enhance hardware efficiency, we employ custom linear approximations of activation functions (hardsigmoid and hardtanh) and fixed-point quantization. Through software-hardware co-design, our 12-nm ASIC implementation operates at 400 MHz, delivering an end-to-end latency of 0.5 ms (equivalent to 2000 Frames Per Second (FPS)) at an energy efficiency of 18.9 $μ$J/frame. JaneEye sets a new benchmark in low-power, high-performance eye-tracking solutions suitable for integration into next-generation XR wearables.

眼动追踪事件相机低功耗硬件加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。