用手机传感器动态校准视线追踪,实时适应用户姿势变化。
MAC-Gaze: Motion-Aware Continual Calibration for Mobile Gaze Tracking
- 结合手机陀螺仪与持续学习,自动识别姿势改变并触发校准。
- 在两个数据集上分别降低19.9%和31.7%的视线误差。
- 适合移动端视线追踪、可穿戴设备等动态场景应用。
移动端视线追踪面临的核心挑战是:用户姿势和设备朝向变化导致精度下降。传统一次性校准方法无法适应动态变化,性能随时间退化。本文提出MAC-Gaze,一种基于运动感知的持续校准方法,利用智能手机惯性测量单元(IMU)传感器与持续学习技术,自动检测用户运动状态变化并更新视线模型。系统融合预训练视觉视线估计器与基于IMU的动作识别模型,通过聚类驱动的混合决策机制,在运动模式显著偏离以往状态时触发重校准。为在学习新运动状态的同时避免灾难性遗忘,采用基于回放的持续学习策略,确保对旧状态的性能保持。我们在公开的RGBDGaze数据集及自建的10小时多模态MotionGaze数据集(含481,000+图像、800,000+IMU读数)上进行大量实验,覆盖坐、站、躺、行走等多种姿势。结果表明,相比传统方法,本方法在RGBDGaze上将视线估计误差从1.73厘米降至1.41厘米(降低19.9%),在MotionGaze上从2.81厘米降至1.92厘米(降低31.7%)。该框架为移动场景下的视线追踪提供了稳健解决方案。
原文摘要 · Abstract (English)
Mobile gaze tracking faces a fundamental challenge: maintaining accuracy as users naturally change their postures and device orientations. Traditional calibration approaches, like one-off, fail to adapt to these dynamic conditions, leading to degraded performance over time. We present MAC-Gaze, a Motion-Aware continual Calibration approach that leverages smartphone Inertial measurement unit (IMU) sensors and continual learning techniques to automatically detect changes in user motion states and update the gaze tracking model accordingly. Our system integrates a pre-trained visual gaze estimator and an IMU-based activity recognition model with a clustering-based hybrid decision-making mechanism that triggers recalibration when motion patterns deviate significantly from previously encountered states. To enable accumulative learning of new motion conditions while mitigating catastrophic forgetting, we employ replay-based continual learning, allowing the model to maintain performance across previously encountered motion conditions. We evaluate our system through extensive experiments on the publicly available RGBDGaze dataset and our own 10-hour multimodal MotionGaze dataset (481K+ images, 800K+ IMU readings), encompassing a wide range of postures under various motion conditions including sitting, standing, lying, and walking. Results demonstrate that our method reduces gaze estimation error by 19.9% on RGBDGaze (from 1.73 cm to 1.41 cm) and by 31.7% on MotionGaze (from 2.81 cm to 1.92 cm) compared to traditional calibration approaches. Our framework provides a robust solution for maintaining gaze estimation accuracy in mobile scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。