用Mamba模型加速视频异常检测，速度远超传统方法。
VADMamba: Exploring State Space Models for Fast Video Anomaly Detection
- 基于Mamba的VQ-MaU架构，结合向量量化与状态空间建模。
- 在三个数据集上实现更快推理速度，准确率不降反升。
- 适合追求实时视频分析的工业场景应用。
视频异常检测（VAD）方法多基于CNN或Transformer，虽精度高但推理慢。状态空间模型如Mamba通过选择性扫描提升计算效率，具备长程建模潜力。本文首次将Mamba引入VAD，提出VADMamba框架，基于多任务学习进行帧预测与光流重建。设计VQ-Mamba Unet（VQ-MaU）结构，融合向量量化（VQ）层与Mamba驱动的非负视觉状态空间（NVSS）块；构建两个独立的VQ-MaU网络，分别预测帧和重构光流，并采用片段级融合评估策略进一步提升性能。在三个基准数据集上的实验验证了该方法的有效性，相比以往工作显著提升推理速度。代码已开源。
原文摘要 · Abstract (English)
Video anomaly detection (VAD) methods are mostly CNN-based or Transformer-based, achieving impressive results, but the focus on detection accuracy often comes at the expense of inference speed. The emergence of state space models in computer vision, exemplified by the Mamba model, demonstrates improved computational efficiency through selective scans and showcases the great potential for long-range modeling. Our study pioneers the application of Mamba to VAD, dubbed VADMamba, which is based on multi-task learning for frame prediction and optical flow reconstruction. Specifically, we propose the VQ-Mamba Unet (VQ-MaU) framework, which incorporates a Vector Quantization (VQ) layer and Mamba-based Non-negative Visual State Space (NVSS) block. Furthermore, two individual VQ-MaU networks separately predict frames and reconstruct corresponding optical flows, further boosting accuracy through a clip-level fusion evaluation strategy. Experimental results validate the efficacy of the proposed VADMamba across three benchmark datasets, demonstrating superior performance in inference speed compared to previous work. Code is available at https://github.com/jLooo/VADMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。