针对快速动作识别不准的问题,提出动态调整注意力窗口的新方法。
VA-AR: Learning Velocity-Aware Action Representations with Mixture of Window Attention

- 根据动作速度自动调节注意力窗口大小,捕捉不同速率下的动作特征。
- 在五个数据集上均达到最优性能,尤其在高速动作上提升显著。
- 适合需要精准识别快慢动作的视频分析场景,如体育赛事分析。
动作识别是人工智能中的关键任务,具有广泛的应用价值。我们对五种主流动作识别方法在五个常用数据集上进行了全面分析,发现一个此前未被重视的现象:随着动作速度增加,现有方法性能普遍下降,严重削弱了其实际应用能力。为此,我们提出速度感知的动作识别框架VA-AR,核心思想是:快速动作(如体操后空翻、羽毛球扣杀)发生在短时窗内,需小窗口捕捉细节变化;而慢速动作(如喝水、擦脸)则需大窗口以覆盖完整上下文。为此,VA-AR采用混合窗口注意力(MoWA)机制,根据动作速度动态调整注意力窗口大小,从而获得速度感知表示,显著提升识别准确率。大量实验表明,该方法在五个数据集上均达到当前最优水平,验证了其在多种动作识别场景下的有效性。
原文摘要 · Abstract (English)
Action recognition is a crucial task in artificial intelligence, with significant implications across various domains. We initially perform a comprehensive analysis of seven prominent action recognition methods across five widely-used datasets. This analysis reveals a critical, yet previously overlooked, observation: as the velocity of actions increases, the performance of these methods variably declines, undermining their robustness. This decline in performance poses significant challenges for their application in real-world scenarios. Building on these findings, we introduce the Velocity-Aware Action Recognition (VA-AR) framework to obtain robust action representations across different velocities. Our principal insight is that rapid actions (e.g., the giant circle backward in uneven bars or a smash in badminton) occur within short time intervals, necessitating smaller temporal attention windows to accurately capture intricate changes. Conversely, slower actions (e.g., drinking water or wiping face) require larger windows to effectively encompass the broader context. VA-AR employs a Mixture of Window Attention (MoWA) strategy, dynamically adjusting its attention window size based on the action's velocity. This adjustment enables VA-AR to obtain a velocity-aware representation, thereby enhancing the accuracy of action recognition. Extensive experiments confirm that VA-AR achieves state-of-the-art performance on the same five datasets, demonstrating VA-AR's effectiveness across a broad spectrum of action recognition scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。