用自回归方法预测第一人称视角注视点，提升AR与辅助技术精度。
ARGaze: Autoregressive Transformers for Online Egocentric Gaze Estimation
- 基于最近注视目标的上下文窗口，用Transformer逐帧预测当前注视点。
- 在多个数据集上达到最新最好性能，尤其在连续动作中表现更优。
- 适合做实时头戴设备注视估计，代码模型将开源。
在线第一人称注视估计从第一人称视频中仅使用过去和当前帧预测佩戴者视线方向，对增强现实和辅助技术至关重要。与第三人称注视估计不同，该任务缺乏明显的头部或眼睛信号，需依赖手物交互和显著场景内容等稀疏间接线索推断当前视觉注意力。我们观察到，在目标导向活动中，注视具有强时间连续性：了解最近的注视位置可为下一步预测提供有力先验。受视觉-语言模型中视觉条件自回归解码启发，我们提出ARGaze，将注视估计重构为序列预测：在每个时间步，通过条件于（i）当前视觉特征和（ii）固定长度的注视上下文窗口（近期注视目标估计），由Transformer解码器预测当前注视点。该设计保证因果性并支持有限资源下的流式推理。我们在多个第一人称基准上实现在线评估下的最先进性能，大量消融实验验证了采用有界注视历史的自回归建模对于鲁棒预测至关重要。我们将公开源代码和预训练模型。
原文摘要 · Abstract (English)
Online egocentric gaze estimation predicts where a camera wearer is looking from first-person video using only past and current frames, a task essential for augmented reality and assistive technologies. Unlike third-person gaze estimation, this setting lacks explicit head or eye signals, requiring models to infer current visual attention from sparse, indirect cues such as hand-object interactions and salient scene content. We observe that gaze exhibits strong temporal continuity during goal-directed activities: knowing where a person looked recently provides a powerful prior for predicting where they look next. Inspired by vision-conditioned autoregressive decoding in vision-language models, we propose ARGaze, which reformulates gaze estimation as sequential prediction: at each timestep, a transformer decoder predicts current gaze by conditioning on (i) current visual features and (ii) a fixed-length Gaze Context Window of recent gaze target estimates. This design enforces causality and enables bounded-resource streaming inference. We achieve state-of-the-art performance across multiple egocentric benchmarks under online evaluation, with extensive ablations validating that autoregressive modeling with bounded gaze history is critical for robust prediction. We will release our source code and pre-trained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。