arXiv:2504.18977cs.CV2025-04

提出3D金字塔网络融合时空特征,提升视频动作识别精度。

3DPyranet Features Fusion for Spatio-temporal Feature Learning

  • 用3D金字塔结构同时学习空间与时间特征,保持生物合理性
  • 在三个基准数据集上超越现有方法,第四组表现相当
  • 参数少、计算量低,适合真实场景中相机抖动环境

卷积神经网络(CNN)通过滑动核在图像上生成输出图,相比全连接网络减少了参数量。尽管传统CNN在手写字符和交通标志识别中表现良好,其深层变体也在目标、场景和动作识别等更复杂任务中取得成功。然而,深层CNN增加了层数和核数,导致参数量上升,部分削弱了原始CNN的参数优势。本文提出一种名为3DPyraNet的三维金字塔神经网络及其判别性时空特征学习方法3DPyraNet-F。3DPyraNet引入新的加权机制,通过分析多个相邻帧,从空间和时间维度共同学习特征,保持输入图像的空间拓扑结构,且参数更少、计算与内存开销更低。3DPyraNet-F提取网络最高层特征图,融合为单一向量,输入线性SVM分类器,显著提升视频中人体动作与动态场景的识别效果。实验表明,在真实环境特别是存在相机运动干扰时,3DPyraNet表现优异;在三个基准数据集上明显优于当前最优方法,第四组结果也具有可比性。

原文摘要 · Abstract (English)

Convolutional neural network (CNN) slides a kernel over the whole image to produce an output map. This kernel scheme reduces the number of parameters with respect to a fully connected neural network (NN). While CNN has proven to be an effective model in recognition of handwritten characters and traffic signal sign boards, etc. recently, its deep variants have proven to be effective in similar as well as more challenging applications like object, scene and action recognition. Deep CNN add more layers and kernels to the classical CNN, increasing the number of parameters, and partly reducing the main advantage of CNN which is less parameters. In this paper, a 3D pyramidal neural network called 3DPyraNet and a discriminative approach for spatio-temporal feature learning based on it, called 3DPyraNet-F, are proposed. 3DPyraNet introduces a new weighting scheme which learns features from both spatial and temporal dimensions analyzing multiple adjacent frames and keeping a biological plausible structure. It keeps the spatial topology of the input image and presents fewer parameters and lower computational and memory costs compared to both fully connected NNs and recent deep CNNs. 3DPyraNet-F extract the features maps of the highest layer of the learned network, fuse them in a single vector, and provide it as input in such a way to a linear-SVM classifier that enhances the recognition of human actions and dynamic scenes from the videos. Encouraging results are reported with 3DPyraNet in real-world environments, especially in the presence of camera induced motion. Further, 3DPyraNet-F clearly outperforms the state-of-the-art on three benchmark datasets and shows comparable result for the fourth.

视频动作识别时空特征3D神经网络轻量化模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。