用记忆回溯实现零样本实时视频异常检测
Flashback: Memory-Driven Zero-shot, Real-time Video Anomaly Detection
- 基于人类记忆机制，离线构建正常与异常的伪场景记忆
- 在线推理时仅靠相似度搜索，实现在消费级GPU上实时处理
- 在UCF-Crime和XD-Violence数据集上分别提升7.0和13.1个百分点
视频异常检测（VAD）可自动识别视频中的异常事件，减少大规模监控部署中对人工操作的依赖。然而，领域依赖性和实时性要求是其实现落地的两大障碍——需对输入视频进行近乎即时的处理。为此，我们提出Flashback，一种零样本、实时的视频异常检测范式。受人类基于过往经验快速判断当前场景异常的认知机制启发，Flashback分为回忆与响应两个阶段：在离线回忆阶段，使用现成的大语言模型（LLM）构建包含正常与异常描述的伪场景记忆，无需真实异常数据；在在线响应阶段，将输入视频片段嵌入并与其记忆进行相似度匹配。通过在推理时完全避免调用LLM，Flashback即使在消费级GPU上也能实现实时处理。在两个来自真实监控场景的大规模数据集UCF-Crime和XD-Violence上，分别取得87.3 AUC（+7.0个百分点）和75.1 AP（+13.1个百分点），显著优于现有零样本VAD方法。
原文摘要 · Abstract (English)
Video Anomaly Detection (VAD) automatically identifies anomalous events from video, mitigating the need for human operators in large-scale surveillance deployments. However, two fundamental obstacles hinder real-world adoption: domain dependency and real-time constraints -- requiring near-instantaneous processing of incoming video. To this end, we propose Flashback, a zero-shot and real-time video anomaly detection paradigm. Inspired by the human cognitive mechanism of instantly judging anomalies and reasoning in current scenes based on past experience, Flashback operates in two stages: Recall and Respond. In the offline recall stage, an off-the-shelf LLM builds a pseudo-scene memory of both normal and anomalous captions without any reliance on real anomaly data. In the online respond stage, incoming video segments are embedded and matched against this memory via similarity search. By eliminating all LLM calls at inference time, Flashback delivers real-time VAD even on a consumer-grade GPU. On two large datasets from real-world surveillance scenarios, UCF-Crime and XD-Violence, we achieve 87.3 AUC (+7.0 pp) and 75.1 AP (+13.1 pp), respectively, outperforming prior zero-shot VAD methods by large margins.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。