通过平滑正则化提升轻量视频模型的时序建模能力
Smooth regularization for efficient video recognition
- 将连续帧间特征变化建模为高斯随机游走，强制中间层输出平滑
- 在Kinetics-600上使轻量模型准确率提升3.8%~6.4%
- 适合追求低计算开销的高效视频识别任务
我们提出一种平滑正则化技术，为视频识别模型引入强时序归纳偏置，尤其适用于轻量级架构。该方法通过将连续帧间特征变化建模为高斯随机游走（GRW），惩罚突变的表征跃迁，从而促进低加速度解，更符合视频固有的时序连贯性。借助这种强制平滑性，轻量模型能更有效捕捉复杂时序动态。应用于此类模型后，在Kinetics-600上实现3.8%至6.4%的准确率提升。值得注意的是，在各自FLOP约束下，使用该正则化的MoViNets模型族相比当前最优结果提升3.8%至6.1%；而MobileNetV3与MoViNets-Stream系列在相近内存占用下相较先前最优模型提升4.9%至6.4%。代码与模型已公开于https://github.com/cmusatyalab/grw-smoothing。
原文摘要 · Abstract (English)
We propose a smooth regularization technique that instills a strong temporal inductive bias in video recognition models, particularly benefiting lightweight architectures. Our method encourages smoothness in the intermediate-layer embeddings of consecutive frames by modeling their changes as a Gaussian Random Walk (GRW). This penalizes abrupt representational shifts, thereby promoting low-acceleration solutions that better align with the natural temporal coherence inherent in videos. By leveraging this enforced smoothness, lightweight models can more effectively capture complex temporal dynamics. Applied to such models, our technique yields a 3.8% to 6.4% accuracy improvement on Kinetics-600. Notably, the MoViNets model family trained with our smooth regularization improves the current state of the art by 3.8% to 6.1% within their respective FLOP constraints, while MobileNetV3 and the MoViNets-Stream family achieve gains of 4.9% to 6.4% over prior state-of-the-art models with comparable memory footprints. Our code and models are available at https://github.com/cmusatyalab/grw-smoothing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。