通过模拟认知推理过程，提升第一视角视频长期动作预测能力。
Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation
- 分两阶段：先提取手物交互语义特征，再用强化学习模拟思考-推理-预测流程。
- 在Ego4D等3个数据集上达到最新最好性能，长程预测准确率显著提升。
- 适合做智能助手、人机交互的开发者，尤其关注意图理解与长期规划。
从第一视角视频中进行长期动作预测对人机交互和辅助技术至关重要，能够预判用户意图以实现主动、上下文感知的AI协助。然而现有方法存在三大缺陷：1）未能充分利用手物交互中的细粒度视觉线索；2）忽略动词与名词间的语义依赖；3）缺乏显式认知推理，限制了泛化性和长期预测能力。为此，我们提出INSIGHT——一种统一的两阶段框架。第一阶段聚焦于从手物交互区域提取语义丰富的特征，并利用动词-名词共现矩阵增强动作表征。第二阶段引入基于强化学习的模块，通过结构化过程模拟显式认知推理：视觉感知（思考）→意图推断（推理）→动作预测（回答）。在Ego4D、EPIC-Kitchens-55和EGTEA Gaze+等多个基准上的大量实验表明，INSIGHT实现了最先进的性能，验证了其有效性和强泛化能力。
原文摘要 · Abstract (English)
Long-term action anticipation from egocentric video is critical for applications such as human-computer interaction and assistive technologies, where anticipating user intent enables proactive and context-aware AI assistance. However, existing approaches suffer from three key limitations: 1) underutilization of fine-grained visual cues from hand-object interactions, 2) neglect of semantic dependencies between verbs and nouns, and 3) lack of explicit cognitive reasoning, limiting generalization and long-term forecasting ability. To overcome these challenges, we propose INSIGHT, a unified two-stage framework for egocentric action anticipation. In the first stage, INSIGHT focuses on extracting semantically rich features from hand-object interaction regions and enhances action representations using a verb-noun co-occurrence matrix. In the second stage, it introduces a reinforcement learning-based module that simulates explicit cognitive reasoning through a structured process: visual perception (think) -> intention inference (reason) -> action anticipation (answer). Extensive experiments on Ego4D, EPIC-Kitchens-55, and EGTEA Gaze+ benchmarks show that INSIGHT achieves state-of-the-art performance, demonstrating its effectiveness and strong generalization capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。