利用视频运动矢量自适应裁剪，无须训练即可提升动作识别效率
MoCrop: Training Free Motion Guided Cropping for Efficient Video Action Recognition
- 基于运动矢量动态定位运动密集区域进行裁剪
- 在UCF101上实现3.5%准确率提升或26.5%计算量减少
- 无需训练、适配多种模型，适合实时系统部署
标准视频动作识别模型通常处理全帧缩放后的图像，存在空间冗余和高计算开销问题。为此，我们提出MoCrop，一个面向压缩域的运动感知自适应裁剪模块，用于高效视频动作识别。该模块利用H.264视频中天然存在的运动矢量（MVs），在推理阶段无需训练或参数更新，即可定位运动密集区域并生成自适应裁剪。其轻量化设计融合三大组件：用于异常值过滤的合并与去噪（MD）、高效的重要性采样蒙特卡洛采样（MCS）以及最优区域定位的运动网格搜索（MGS）。MoCrop可作为通用“即插即用”模块适配多种主干网络。在UCF101上的大量实验表明，它兼具加速与增效作用：使用ResNet-50时，在等价浮点运算量下准确率提升3.5%（注意力设置），或在减少26.5%浮点运算量的同时提升2.4%准确率（效率设置）；应用于CoViAR时，准确率达89.2%，计算量从11.6降至8.5 GFLOPs（约减少27%）。在MobileNet-V3、EfficientNet-B1和Swin-B上的持续增益验证了其强泛化能力，适用于实时部署。代码与模型已公开于https://github.com/microa/MoCrop。
原文摘要 · Abstract (English)
Standard video action recognition models often process typically resized full frames, suffering from spatial redundancy and high computational costs. To address this, we introduce MoCrop, a motion-aware adaptive cropping module designed for efficient video action recognition in the compressed domain. Leveraging Motion Vectors (MVs) naturally available in H.264 video, MoCrop localizes motion-dense regions to produce adaptive crops at inference without requiring any training or parameter updates. Our lightweight pipeline synergizes three key components: Merge & Denoise (MD) for outlier filtering, Monte Carlo Sampling (MCS) for efficient importance sampling, and Motion Grid Search (MGS) for optimal region localization. This design allows MoCrop to serve as a versatile "plug-and-play" module for diverse backbones. Extensive experiments on UCF101 demonstrate that MoCrop serves as both an accelerator and an enhancer. With ResNet-50, it achieves a +3.5% boost in Top-1 accuracy at equivalent FLOPs (Attention Setting), or a +2.4% accuracy gain with 26.5% fewer FLOPs (Efficiency Setting). When applied to CoViAR, it improves accuracy to 89.2% or reduces computation by roughly 27% (from 11.6 to 8.5 GFLOPs). Consistent gains across MobileNet-V3, EfficientNet-B1, and Swin-B confirm its strong generality and suitability for real-time deployment. Our code and models are available at https://github.com/microa/MoCrop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。