用合成数据+真实数据蒸馏，实现轻量级设备端高精度眼动追踪。
Rapidly deploying on-device eye tracking by distilling visual foundation models
- 通过合成数据与真实图像联合训练专用教师模型
- 在2000人数据集上误差降低58.6%，模型仅256K参数
- 适合需要快速适配新硬件的眼动追踪系统
眼动追踪（ET）在增强现实和虚拟现实应用中至关重要。然而，针对新产品快速部署高精度、设备端的注视估计仍具挑战性，因硬件配置（如摄像头位置、姿态、光照）常随设备迭代而变化。视觉基础模型（VFMs）在自然图像基准上表现优异，为快速训练与部署提供了可能；但我们发现现成的VFMs在专用近眼红外图像上仍难以达到高精度。为此，我们提出DistillGaze框架，利用带标签的合成数据与无标签真实数据对视觉基础模型进行蒸馏，实现快速、高精度的设备端注视估计。该框架分两阶段：首先，使用合成注视标签和无标签真实图像将VFM转化为领域专用教师模型；合成数据提供可扩展的高质量监督，无标签真实数据弥合合成到真实的数据域差距。其次，基于教师指导与自训练，训练轻量级设备端学生模型。在涵盖超过2000名参与者的大型众包数据集上评估，DistillGaze相比仅用合成数据的基线，中位数注视误差降低58.6%，同时保持256K参数的小型模型，适合实时设备端部署。更广泛地，DistillGaze为适应硬件变化的ET模型训练与部署提供了高效路径，并为结合合成监督与无标签真实数据的设备端回归任务提供了范式。
原文摘要 · Abstract (English)
Eye tracking (ET) plays a critical role in augmented and virtual reality applications. However, rapidly deploying high-accuracy, on-device gaze estimation for new products remains challenging because hardware configurations (e.g., camera placement, camera pose, and illumination) often change across device generations. Visual foundation models (VFMs) excel on natural-image benchmarks and offer a promising path to rapid training and deployment; yet, we find that off-the-shelf VFMs still struggle to reach high accuracy on specialized near-eye infrared images. To close this gap, we introduce DistillGaze, a framework that distills a VFM using labeled synthetic data and unlabeled real data for rapid, high-accuracy on-device gaze estimation. DistillGaze proceeds in two stages. First, we adapt a VFM into a domain-specialized teacher using synthetic gaze labels and unlabeled real images. Synthetic data provide scalable, high-quality gaze supervision, while unlabeled real data bridges the synthetic-to-real domain gap. Second, we train an on-device student from both teacher guidance and self-training. Evaluated on a large-scale crowd-sourced dataset spanning more than 2,000 participants, DistillGaze reduces median gaze error by 58.6% relative to synthetic-only baselines while maintaining a lightweight 256K-parameter model suitable for real-time on-device deployment. More broadly, DistillGaze offers an efficient path to training and deploying ET models that adapt to hardware changes, and a recipe for combining synthetic supervision with unlabeled real data in on-device regression tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。