用快慢双通道检测视频异常,提升准确率与可解释性。
SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model
- 快通道粗检,慢通道精析,仅处理模糊片段以降开销。
- 在4个数据集上准确率超基线,计算量大幅降低。
- 适合对可靠性要求高的真实场景视频监控应用。
视频异常检测(VAD)旨在识别视频中的意外事件,在安全关键领域有广泛应用。尽管仅用正常样本训练的半监督方法已受关注,但常面临误报率高、可解释性差的问题。近期视觉语言模型(VLM)展现出强大的多模态推理能力,为可解释异常检测带来新机遇,但其高计算成本与缺乏领域适配性阻碍了实时部署与可靠性。受人类视觉感知中快速与缓慢通路互补的启发,本文提出SlowFastVAD,一种融合快速异常检测器与慢速检测器(即检索增强生成增强的VLM)的混合框架。具体而言,快速检测器先输出粗略异常置信度,仅对少量模糊片段由更慢但更可解释的VLM进行深入分析与推理。此外,为使VLM适应特定领域,构建知识库,包含基于少量正常样本的正常模式及由VLM推断出的异常模式。推理时,通过检索相关模式并增强提示,提升异常推理效果。最后,平滑融合快慢检测器的异常置信度,增强检测鲁棒性。在四个基准数据集上的大量实验表明,SlowFastVAD有效结合两者优势,实现显著更高的检测准确率与可解释性,同时大幅降低计算开销,适用于高可靠性要求的真实世界视频异常检测场景。
原文摘要 · Abstract (English)
Video anomaly detection (VAD) aims to identify unexpected events in videos and has wide applications in safety-critical domains. While semi-supervised methods trained on only normal samples have gained traction, they often suffer from high false alarm rates and poor interpretability. Recently, vision-language models (VLMs) have demonstrated strong multimodal reasoning capabilities, offering new opportunities for explainable anomaly detection. However, their high computational cost and lack of domain adaptation hinder real-time deployment and reliability. Inspired by dual complementary pathways in human visual perception, we propose SlowFastVAD, a hybrid framework that integrates a fast anomaly detector with a slow anomaly detector (namely a retrieval augmented generation (RAG) enhanced VLM), to address these limitations. Specifically, the fast detector first provides coarse anomaly confidence scores, and only a small subset of ambiguous segments, rather than the entire video, is further analyzed by the slower yet more interpretable VLM for elaborate detection and reasoning. Furthermore, to adapt VLMs to domain-specific VAD scenarios, we construct a knowledge base including normal patterns based on few normal samples and abnormal patterns inferred by VLMs. During inference, relevant patterns are retrieved and used to augment prompts for anomaly reasoning. Finally, we smoothly fuse the anomaly confidence of fast and slow detectors to enhance robustness of anomaly detection. Extensive experiments on four benchmarks demonstrate that SlowFastVAD effectively combines the strengths of both fast and slow detectors, and achieves remarkable detection accuracy and interpretability with significantly reduced computational overhead, making it well-suited for real-world VAD applications with high reliability requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。