用Mamba模型高效学习视频正常模式,提升异常检测精度。
STNMamba: Mamba-based Spatial-Temporal Normality Learning for Video Anomaly Detection
- 设计双分支编码器,分别提取多尺度外观和运动特征。
- 通过融合模块与记忆库,建模时空一致性,抑制异常表示。
- 参数少、计算量低,在多个数据集上表现优异。
视频异常检测(VAD)在智能视频系统中具有重要应用价值。现有基于CNN和Transformer的方法仍面临计算开销大、时空正常模式学习不足的问题。近期,Mamba模型以线性复杂度捕捉长程依赖,展现出巨大潜力。为此,本文提出轻量高效的Mamba-based网络STNMamba,通过精心设计的Mamba模块增强时空正常模式学习。首先,采用双编码器结构:空间编码器使用多尺度视觉状态块(MS-VSSB)提取多尺度外观特征,时间编码器采用通道感知视觉状态块(CA-VSSB)捕捉显著运动模式。其次,引入时空交互模块(STIM),通过时空融合块(STFB)在多层级融合时空信息,并利用记忆库存储正常模式原型,限制模型对异常的表达能力。在三个基准数据集上的大量实验表明,所提方法在参数更少、计算成本更低的前提下,达到具有竞争力的性能。
原文摘要 · Abstract (English)
Video anomaly detection (VAD) has been extensively researched due to its potential for intelligent video systems. However, most existing methods based on CNNs and transformers still suffer from substantial computational burdens and have room for improvement in learning spatial-temporal normality. Recently, Mamba has shown great potential for modeling long-range dependencies with linear complexity, providing an effective solution to the above dilemma. To this end, we propose a lightweight and effective Mamba-based network named STNMamba, which incorporates carefully designed Mamba modules to enhance the learning of spatial-temporal normality. Firstly, we develop a dual-encoder architecture, where the spatial encoder equipped with Multi-Scale Vision Space State Blocks (MS-VSSB) extracts multi-scale appearance features, and the temporal encoder employs Channel-Aware Vision Space State Blocks (CA-VSSB) to capture significant motion patterns. Secondly, a Spatial-Temporal Interaction Module (STIM) is introduced to integrate spatial and temporal information across multiple levels, enabling effective modeling of intrinsic spatial-temporal consistency. Within this module, the Spatial-Temporal Fusion Block (STFB) is proposed to fuse the spatial and temporal features into a unified feature space, and the memory bank is utilized to store spatial-temporal prototypes of normal patterns, restricting the model's ability to represent anomalies. Extensive experiments on three benchmark datasets demonstrate that our STNMamba achieves competitive performance with fewer parameters and lower computational costs than existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。