arXiv:2603.25467cs.CV2026-03中稿 · the Large-scale Vi…

用分层网格+语言模型生成异常候选,再精准定位分割异常区域。

GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids

  • 分层网格输入让语言模型生成开放集异常描述,避免直接检测失效。
  • 在UCSD Ped2上像素级AUROC达77.59,超越部分微调模型。
  • 无需训练即可生成像素级掩码,适合快速部署到新场景。

视觉语言模型(VLM)具备强大的开放集推理能力,但直接用于视频监控异常检测时易产生漏检或误报。问题不在于模型本身,而在于使用方式。我们主张将VLM作为异常提案者,生成自然语言异常描述,再由专用的空间时间模块进行定位与追踪。为此提出GridVAD:一个无需训练的流水线,可生成像素级异常掩码。通过分层帧网格表示,VLM生成异常提案;自一致性整合(SCC)通过多次采样后重复出现的提案过滤幻觉;Grounding DINO将有效提案锚定为边界框,SAM2将其传播为密集掩码。每片段仅需固定M+1次调用,不受视频长度影响。在UCSD Ped2上,像素级AUROC达77.59,优于部分微调模型TAO(75.11),且对象级RBDC性能较其他零样本方法提升超5倍。消融实验表明SCC可在小幅降低对象召回率下显著提升像素级指标。效率测试显示,相比均匀每帧调用VLM,GridVAD更高效,且额外输出密集掩码。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) are powerful open-set reasoners, yet their direct use as anomaly detectors in video surveillance is fragile: without calibrated anomaly priors, they alternate between missed detections and hallucinated false alarms. We argue the problem is not the VLM itself but how it is used. VLMs should function as anomaly proposers, generating open-set candidate descriptions that are then grounded and tracked by purpose-built spatial and temporal modules. We instantiate this propose-ground-propagate principle in GridVAD, a training-free pipeline that produces pixel-level anomaly masks without any domain-specific training. A VLM reasons over stratified grid representations of video clips to generate natural-language anomaly proposals. Self-Consistency Consolidation (SCC) filters hallucinations by retaining only proposals that recur across multiple independent samplings. Grounding DINO anchors each surviving proposal to a bounding box, and SAM2 propagates it as a dense mask through the anomaly interval. The per-clip model budget is fixed at M+1 calls regardless of video length, where M can be set according to the proposals needed. On UCSD Ped2, GridVAD achieves the highest Pixel-AUROC (77.59) among all compared methods, surpassing even the partially fine-tuned TAO (75.11) and outperforms other zero-shot approaches on object-level RBDC by over 5x. Ablations reveal that SCC provides a controllable precision-recall tradeoff: filtering improves all pixel level metrics at a modest cost in object-level recall. Efficiency experiments show GridVAD is 2.7x more call-efficient than uniform per-frame VLM querying while additionally producing dense segmentation masks.Code and qualitative video results are available at https://gridvad.github.io.

异常检测视频理解零样本语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。