EAST让模型提前预测动作，仅靠部分视频帧就能准确判断。
EAST: Early Action Prediction Sampling Strategy with Token Masking
- 随机采样观察与未观察帧的时间点，统一训练不同观察比例。
- 联合学习当前和未来帧特征，提升模型性能，编码器也可超群。
- 用令牌掩码减少内存占用，训练提速2倍，精度几乎不变。
早期动作预测需在动作未完全展开时做出判断，但视觉信息有限使其尤为困难。本文提出EAST框架，通过随机采样观测与未观测视频帧之间的时间步，使单一模型能无缝泛化于测试时任意观察比例。实证研究发现，联合学习当前与未来（理想）表示可显著提升性能，甚至使纯编码器模型表现优异。为提升可扩展性，提出令牌掩码方法，将内存使用减半，训练速度提升2倍，精度损失可忽略。结合预测解码器，EAST在NTU60、SSv2和UCF101上分别超越此前最佳结果10.1、7.7和3.9个百分点，刷新了当前最佳性能。
原文摘要 · Abstract (English)
Early action prediction seeks to anticipate an action before it fully unfolds, but limited visual evidence makes this task especially challenging. We introduce EAST, a simple and efficient framework that enables a model to reason about incomplete observations. In our empirical study, we identify key components when training early action prediction models. Our key contribution is a randomized training strategy that samples a time step separating observed and unobserved video frames, enabling a single model to generalize seamlessly across all test-time observation ratios. We further show that joint learning on both observed and future (oracle) representations significantly boosts performance, even allowing an encoder-only model to excel. To improve scalability, we propose a token masking procedure that cuts memory usage in half and accelerates training by 2x with negligible accuracy loss. Combined with a forecasting decoder, EAST sets a new state of the art on NTU60, SSv2, and UCF101, surpassing previous best work by 10.1, 7.7, and 3.9 percentage points, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。