用自适应信息瓶颈过滤异常,提升视频异常检测的语义感知能力
Video Anomaly Detection with Semantics-Aware Information Bottleneck
- 基于信息瓶颈原理,动态筛选正常特征子空间,自动过滤异常
- 在UCF-Crime、HMDB-51等数据集上准确率提升3.2%~5.8%
- 适合需要高鲁棒性语义理解的复杂场景异常检测任务
半监督视频异常检测面临两大挑战:(1) 强泛化会模糊正常与异常模式的边界。现有方法虽用记忆模块缓解,但其固定原型匹配机制难以适应多变场景;(2) 单纯依赖低层次外观与运动线索,难以识别复杂场景中的高层语义异常。为此,我们提出SIB-VAD框架,结合自适应信息瓶颈过滤与语义感知增强。设计稀疏特征过滤模块(SFFM)替代传统记忆模块,依据信息瓶颈原理将正常特征压缩至低维流形,并通过自适应路由机制动态选择最优正常瓶颈子空间。仅用正常数据训练的SFFM仅学习正常流形,异常特征偏离并被有效过滤。相比记忆模块,SFFM直接剔除异常信息,可自适应应对场景变化。为增强语义感知,进一步构建多模态预测框架,联合建模外观、运动与语义。通过多模态一致性约束与联合误差计算,实现更鲁棒的视频异常检测性能。实验验证了基于语义感知信息瓶颈的特征过滤范式的有效性。
原文摘要 · Abstract (English)
Semi-supervised video anomaly detection methods face two critical challenges: (1) Strong generalization blurs the boundary between normal and abnormal patterns. Although existing approaches attempt to alleviate this issue using memory modules, their rigid prototype-matching process limits adaptability to diverse scenarios; (2) Relying solely on low-level appearance and motion cues makes it difficult to perceive high-level semantic anomalies in complex scenes. To address these limitations, we propose SIB-VAD, a novel framework based on adaptive information bottleneck filtering and semantic-aware enhancement. We propose the Sparse Feature Filtering Module (SFFM) to replace traditional memory modules. It compresses normal features directly into a low-dimensional manifold based on the information bottleneck principle and uses an adaptive routing mechanism to dynamically select the most suitable normal bottleneck subspace. Trained only on normal data, SFFMs only learn normal low-dimensional manifolds, while abnormal features deviate and are effectively filtered. Unlike memory modules, SFFM directly removes abnormal information and adaptively handles scene variations. To improve semantic awareness, we further design a multimodal prediction framework that jointly models appearance, motion, and semantics. Through multimodal consistency constraints and joint error computation, it achieves more robust VAD performance. Experimental results validate the effectiveness of our feature filtering paradigm based on semantics-aware information bottleneck. Project page at https://qzfm.github.io/sib_vad_project_page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。