融合3D CNN与Transformer，提升视频行为识别准确率
A Framework Combining 3D CNN and Transformer for Video-Based Behavior Recognition
- 用3D CNN提取局部时空特征，Transformer捕捉长时依赖
- 在多个基准数据集上准确率优于纯CNN或纯Transformer
- 适合需要高精度视频理解的安防与交互系统
基于视频的行为识别在公共安全、智能监控和人机交互等领域至关重要。传统3D卷积神经网络（3D CNN）能有效捕捉局部时空特征，但难以建模长距离依赖；而Transformer擅长学习全局上下文信息，却面临计算成本高的问题。为此，我们提出一种结合3D CNN与Transformer架构的混合框架：3D CNN模块提取低层时空特征，Transformer模块捕捉长时序依赖，并通过融合机制整合两者表示。在多个基准数据集上的评估表明，该模型在保持可管理复杂度的前提下，性能优于传统的3D CNN和独立的Transformer。消融实验进一步验证了两种模块的互补优势。该混合框架为视频行为识别提供了高效且可扩展的解决方案。
原文摘要 · Abstract (English)
Video-based behavior recognition is essential in fields such as public safety, intelligent surveillance, and human-computer interaction. Traditional 3D Convolutional Neural Network (3D CNN) effectively capture local spatiotemporal features but struggle with modeling long-range dependencies. Conversely, Transformers excel at learning global contextual information but face challenges with high computational costs. To address these limitations, we propose a hybrid framework combining 3D CNN and Transformer architectures. The 3D CNN module extracts low-level spatiotemporal features, while the Transformer module captures long-range temporal dependencies, with a fusion mechanism integrating both representations. Evaluated on benchmark datasets, the proposed model outperforms traditional 3D CNN and standalone Transformers, achieving higher recognition accuracy with manageable complexity. Ablation studies further validate the complementary strengths of the two modules. This hybrid framework offers an effective and scalable solution for video-based behavior recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。