arXiv:2506.03162cs.CVcs.AI2025-06中稿 · the 22nd IEEE Inte…被引 5

用双分支Mamba模型提升暴力行为检测效率与准确率。

FuseMamba-VD: Dual Branch VideoMamba with Gated Class Token Fusion for Violence Detection

  • 双分支设计分别提取空间与时间特征,通过门控机制融合增强判别力。
  • 在多数据集混合基准上达到最新最好性能,参数量仅12.8M。
  • 适合需要低延迟、高效率的实时监控场景使用。

监控摄像头的普及推动了自动化暴力行为检测的需求。尽管CNN和Transformer在时空特征提取上表现良好,但在长时依赖建模和计算效率方面存在瓶颈。本文提出FuseMamba-VD:一种基于状态空间模型(SSM)的双分支VideoMamba架构,其中一分支捕捉空间特征,另一分支专注时间动态。通过门控类令牌融合(GCTF)机制实现连续特征融合,显著提升复杂监控场景下的暴力行为检测能力。我们构建了一个新基准,整合RWF-2000、RLVS、SURV和VioPeru数据集,并严格划分训练与测试集。实验表明,该模型在该基准及近期提出的DVD数据集上均取得领先性能,参数量仅12.8M,兼具高精度与低计算开销,验证了SSM在可扩展、资源高效视频检测中的潜力。代码与预训练模型已开源。

原文摘要 · Abstract (English)

The rapid proliferation of surveillance cameras has increased the demand for automated violence detection. While CNNs and Transformers have shown success in extracting spatio-temporal features, they struggle with long-term dependencies and computational efficiency. We propose FuseMamba-VD: Dual Branch VideoMamba with Gated Class Token Fusion (GCTF), an efficient architecture combining a dual-branch design and a state-space model (SSM) backbone where one branch captures spatial features, while the other focuses on temporal dynamics. The model performs continuous fusion via a gating mechanism from the spatial branch into the temporal branch to enhance detection of violent activities even in challenging surveillance scenarios. We also present a new benchmark by merging RWF-2000, RLVS, SURV and VioPeru datasets in video violence detection, ensuring strict separation between training and testing sets. Experimental results demonstrate that our model achieves state-of-the-art performance on this benchmark and also on DVD dataset which is a recently introduced dataset on video violence detection, offering an optimal balance between accuracy and computational efficiency, demonstrating the promise of SSMs for scalable, resource efficient video violence detection. The code and pre-trained models are available at https://github.com/damith92/FuseMamba-VD.

暴力检测Mamba视频分析轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。