提出一种高效3D CNN，用分组卷积提升视频动作识别性能
An Efficient 3D Convolutional Neural Network with Channel-wise, Spatial-grouped, and Temporal Convolutions
- 采用通道-空间分组-时间三级卷积结构，降低计算开销
- 在多个数据集上以更小模型达到或超越大模型性能
- 仅用RGB输入即超越双流方法，适合资源受限场景
近年来视频动作识别取得显著进展，但许多工作依赖ImageNet预训练的2D主干网络进行微调，限制了视频识别模型的效率提升。本文提出一种简单高效的3D卷积神经网络用于视频动作识别，其构建模块包含通道卷积、空间分组卷积和时间卷积。我们在不依赖预训练的情况下，直接在目标数据集上训练该网络。在Something-Something-V1&V2、Kinetics-400和Multi-Moments in Time数据集上，该网络性能与甚至超过数倍更大的模型；在细粒度动作识别数据集FineGym上，仅使用RGB输入即比此前基于双流方法的最先进准确率高出5%以上。
原文摘要 · Abstract (English)
There has been huge progress on video action recognition in recent years. However, many works focus on tweaking existing 2D backbones due to the reliance of ImageNet pretraining, which restrains the models from achieving higher efficiency for video recognition. In this work we introduce a simple and very efficient 3D convolutional neural network for video action recognition. The design of the building block consists of a channel-wise convolution, followed by a spatial group convolution, and finally a temporal convolution. We evaluate the performance and efficiency of our proposed network on several video action recognition datasets by directly training on the target dataset without relying on pertaining. On Something-Something-V1&V2, Kinetics-400 and Multi-Moments in Time, our network can match or even surpass the performance of other models which are several times larger. On the fine-grained action recognition dataset FineGym, we beat the previous state-of-the-art accuracy achieved with 2-stream methods by more than 5% using only RGB input.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。