用自然语言动态定义异常,让视频检测适应变化场景。
Language-guided Open-world Video Anomaly Detection under Weak Supervision
- 通过自然语言在推理时灵活定义异常行为。
- 在7个数据集上实现零样本最优性能,准确率超现有方法。
- 适合需要实时调整异常标准的安防、医疗等场景。
视频异常检测(VAD)旨在识别偏离预期的异常事件。在开放世界中,预期事件会随需求变化——例如流感期间不戴口罩被视为异常,平时则不然。现有方法假设异常定义固定,无法适应此类变化。为此,我们提出一种新型开放世界VAD范式,支持通过用户提供的自然语言在推理时动态定义异常。该范式需建立视频与文本定义到异常得分的鲁棒映射,因此我们提出LaGoVAD模型,在弱监督下通过两种正则化策略实现自适应:动态视频合成以多样化异常持续时间,对比学习结合负样本挖掘增强特征鲁棒性。训练该模型需多样化的异常定义,但现有数据集通常仅有标签无语义描述。为此,我们构建了目前最大最多样化的视频异常数据集PreVAD,包含35,279段标注视频,具备多层级类别标签与明确的异常定义描述。在七个数据集上的零样本实验表明,LaGoVAD达到当前最优性能。代码与数据已开源。
原文摘要 · Abstract (English)
Video anomaly detection (VAD) aims to detect anomalies that deviate from what is expected. In open-world scenarios, the expected events may change as requirements change. For example, not wearing a mask may be considered abnormal during a flu outbreak but normal otherwise. However, existing methods assume that the definition of anomalies is invariable, and thus are not applicable to the open world. To address this, we propose a novel open-world VAD paradigm with variable definitions, allowing guided detection through user-provided natural language at inference time. This paradigm necessitates establishing a robust mapping from video and textual definition to anomaly scores. Therefore, we propose LaGoVAD (Language-guided Open-world Video Anomaly Detector), a model that dynamically adapts anomaly definitions under weak supervision with two regularization strategies: diversifying the relative durations of anomalies via dynamic video synthesis, and enhancing feature robustness through contrastive learning with negative mining. Training such adaptable models requires diverse anomaly definitions, but existing datasets typically provide labels without semantic descriptions. To bridge this gap, we collect PreVAD (Pre-training Video Anomaly Dataset), the largest and most diverse video anomaly dataset to date, featuring 35,279 annotated videos with multi-level category labels and descriptions that explicitly define anomalies. Zero-shot experiments on seven datasets demonstrate LaGoVAD's SOTA performance. Our dataset and code are released at https://github.com/Kamino666/LaGoVAD-PreVAD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。