让视觉语言模型看懂视频动态,实时生成连贯异常解释。
LATERN: Test-Time Context-Aware Explainable Video Anomaly Detection

- 用记忆机制融合历史帧与图文对齐信息,提升异常评分可靠性。
- 通过递归聚合实现事件级判断,准确率在UCF-Crime上达87.3%。
- 适合需要可解释性视频异常检测的安防与内容审核场景。
视觉语言模型(VLM)凭借强大的视觉推理能力与自然语言解释性,成为视频异常检测(VAD)的新范式。然而,现有方法因令牌限制独立处理片段,缺乏结构化时序上下文,导致预测碎片化。本文提出一种上下文感知框架LATERN,将VAD重构为时序证据聚合过程。其包含两个模块:基于图像记忆的异常评分(CEA)通过帧多样性与图文对齐选择历史内容作为扩展上下文,提升评分可靠性;在此基础上,递归证据聚合(REA)实现时序递归聚合,识别连贯异常区间,生成事件级决策与基于视觉-文本证据的解释。在UCF-Crime与XD-Violence等挑战性基准上,LATERN显著提升冻结VLM在测试阶段的检测准确率与解释一致性,生成时序连贯、语义可信的事件级解释。
原文摘要 · Abstract (English)
Vision-language models (VLMs) have recently emerged as a promising paradigm for video anomaly detection (VAD) due to their strong visual reasoning ability and natural language-based explainability. In this paper, we aim to address a key limitation of such pipelines, which perform segment-level inference independently owing to token constraints and reason without structured temporal context, allowing VLMs to interpret anomalies as deviations from evolving video dynamics rather than producing fragmented predictions and explanations. To specify, we propose a context-aware framework named LATERN, which reformulates VAD as a temporal evidence aggregation process. LATERN consists of two complementary modules: Context-Aware Anomaly Scoring (CEA) and Recursive Evidence Aggregation (REA). CEA introduces a novel image-grounded memory mechanism, which selectively chooses historical content via frame diversity and visual-textual alignment as expanded context to help generate reliable anomaly scores. Building upon these scores, REA performs recursive temporal aggregation to identify coherent anomaly intervals and produce event-level decisions and explanations grounded in visual-textual evidence. Extensive experiments on challenging benchmarks, including UCF-Crime and XD-Violence, show that LATERN enhances detection accuracy and explanation consistency for frozen VLMs during test time, while generating temporally coherent and semantically grounded event-level explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。