在高分辨率视频分类中，注意力机制能有效弥补时间信息丢失带来的性能下降。
Effects of Different Attention Mechanisms Applied on 3D Models in Video Classification
- 在3D CNN模型中引入多种注意力模块，增强对时间特征的捕捉能力。
- 多头注意力使改进版R(2+1)D在UCF101上达到88.98%准确率。
- 适合关注视频动作识别中时间建模与注意力设计的研究者。
人体动作识别因应用广泛已成为计算机视觉重要研究方向。基于3D ResNet的MC3、R3D和R(2+1)D模型通过不同卷积核提取时空特征。本文研究在提升帧分辨率的同时减少时间信息捕获的影响。为此，我们对三类原始模型进行相似改造，并在最终分类器前加入丢弃层；随后为每种模型构建十种新版本，引入卷积块注意力模块（CBAM）、时序卷积网络（TCN）及多头注意力、通道注意力等机制。目的是观察各注意力模块对受限时间信息模型性能的影响。在UCF101数据集上的测试结果显示，添加多头注意力的改进版R(2+1)D达到88.98%准确率。论文指出，时间特征缺失显著影响高分辨率模型表现，且不同注意力变体在类别级准确率上表现出异质性，尽管整体性能相近。
原文摘要 · Abstract (English)
Human action recognition has become an important research focus in computer vision due to the wide range of applications where it is used. 3D Resnet-based CNN models, particularly MC3, R3D, and R(2+1)D, have different convolutional filters to extract spatiotemporal features. This paper investigates the impact of reducing the captured knowledge from temporal data, while increasing the resolution of the frames. To establish this experiment, we created similar designs to the three originals, but with a dropout layer added before the final classifier. Secondly, we then developed ten new versions for each one of these three designs. The variants include special attention blocks within their architecture, such as convolutional block attention module (CBAM), temporal convolution networks (TCN), in addition to multi-headed and channel attention mechanisms. The purpose behind that is to observe the extent of the influence each of these blocks has on performance for the restricted-temporal models. The results of testing all the models on UCF101 have shown accuracy of 88.98% for the variant with multiheaded attention added to the modified R(2+1)D. This paper concludes the significance of missing temporal features in the performance of the newly created increased resolution models. The variants had different behavior on class-level accuracy, despite the similarity of their enhancements to the overall performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。