arXiv:2508.18463cs.CVcs.LG2025-08被引 1

用视觉语言模型和预测机制,让系统在没看过异常时也能识别监控中的异常行为。

Context-Aware Zero-Shot Anomaly Detection in Surveillance Using Contrastive and Predictive Spatiotemporal Modeling

  • 融合时间建模与语义提示,通过预测未来特征发现异常。
  • 在UCF-Crime、HKU-Mall数据集上达到82.3%和76.5%的AUC。
  • 适合需要快速部署、无异常样本的智能监控场景。

由于异常事件的不可预测性和上下文依赖性,监控视频中的异常检测极具挑战。本文提出一种新型上下文感知的零样本异常检测框架,可在训练中未接触异常样本的情况下识别异常行为。该混合架构结合TimeSformer、DPC和CLIP,分别建模时空动态与语义上下文。TimeSformer作为视觉主干提取丰富的时空特征,DPC预测未来表示以识别时间偏差;同时,基于CLIP的语义流通过场景特定文本提示实现概念级异常检测。各组件联合使用InfoNCE与CPC损失进行训练,对齐视觉输入与其时空和语义表示。引入上下文门控机制,通过场景感知线索或全局视频特征调制预测结果。该系统通过结合预测建模与视觉语言理解,在复杂环境中实现对未见行为的泛化能力。代码已开源:https://github.com/NK-II/Context-Aware-Zero-Shot-Anomaly-Detection-in-Surveillance。

原文摘要 · Abstract (English)

Detecting anomalies in surveillance footage is inherently challenging due to their unpredictable and context-dependent nature. This work introduces a novel context-aware zero-shot anomaly detection framework that identifies abnormal events without exposure to anomaly examples during training. The proposed hybrid architecture combines TimeSformer, DPC, and CLIP to model spatiotemporal dynamics and semantic context. TimeSformer serves as the vision backbone to extract rich spatial-temporal features, while DPC forecasts future representations to identify temporal deviations. Furthermore, a CLIP-based semantic stream enables concept-level anomaly detection through context-specific text prompts. These components are jointly trained using InfoNCE and CPC losses, aligning visual inputs with their temporal and semantic representations. A context-gating mechanism further enhances decision-making by modulating predictions with scene-aware cues or global video features. By integrating predictive modeling with vision-language understanding, the system can generalize to previously unseen behaviors in complex environments. This framework bridges the gap between temporal reasoning and semantic context in zero-shot anomaly detection for surveillance. The code for this research has been made available at https://github.com/NK-II/Context-Aware-Zero-Shot-Anomaly-Detection-in-Surveillance.

异常检测零样本视觉语言监控系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。