用伪异常训练，让大模型零样本检测视频异常
No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detection
- 用伪异常数据增强模型泛化能力，无需真实异常样本
- 结合多模态大模型提升语义理解，零样本下达到顶尖性能
- 通过逆注意力压缩令牌，缓解异常时空稀缺问题
视频异常数据因罕见且时空分布稀疏，长期难以获取与检测。现有方法在开放世界场景下表现不佳，主要受限于数据集多样性不足及对上下文依赖的异常语义理解不足。为此，本文提出LAVIDA——一种端到端的零样本视频异常检测框架：首先设计异常暴露采样器，将分割出的物体转为伪异常以增强模型对未见异常类别的适应性；其次引入多模态大语言模型（MLLM），强化语义理解能力；此外，提出基于反向注意力的令牌压缩方法，应对异常模式时空稀缺问题并降低计算开销。整个训练过程仅使用伪异常数据，不依赖真实异常标注。在四个基准数据集上的评估表明，LAVIDA在帧级和像素级异常检测任务中均实现零样本设置下的最优表现。
原文摘要 · Abstract (English)
The collection and detection of video anomaly data has long been a challenging problem due to its rare occurrence and spatio-temporal scarcity. Existing video anomaly detection (VAD) methods under perform in open-world scenarios. Key contributing factors include limited dataset diversity, and inadequate understanding of context-dependent anomalous semantics. To address these issues, i) we propose LAVIDA, an end-to-end zero-shot video anomaly detection framework. ii) LAVIDA employs an Anomaly Exposure Sampler that transforms segmented objects into pseudo-anomalies to enhance model adaptability to unseen anomaly categories. It further integrates a Multimodal Large Language Model (MLLM) to bolster semantic comprehension capabilities. Additionally, iii) we design a token compression approach based on reverse attention to handle the spatio-temporal scarcity of anomalous patterns and decrease computational cost. The training process is conducted solely on pseudo anomalies without any VAD data. Evaluations across four benchmark VAD datasets demonstrate that LAVIDA achieves SOTA performance in both frame-level and pixel-level anomaly detection under the zero-shot setting. Our code is available in https://github.com/VitaminCreed/LAVIDA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。