用视觉语言模型零样本识别监控视频中的异常行为，探索空间推理能力边界。
Evaluation of Vision-LLMs in Surveillance Video
- 将视频转为文本描述，通过语义蕴含评分实现零样本异常检测。
- 小模型在简单事件上表现良好，但遮挡与隐私处理会显著降低准确率。
- 适合关注无监督视频理解、隐私保护与通用感知的开发者和研究者。
社会中摄像头的广泛应用产生了海量视频数据，远超人工监控能力，威胁公共安全。及时发现异常或犯罪事件对响应与预防至关重要。智能体识别意外事件的能力与其空间推理能力密切相关。本文将异常动作识别视为零样本、语言引导的任务，研究视觉语言模型（VLMs）在从稀疏2D视频理解动态3D场景时的空间推理能力。具体而言，评估小型预训练视觉-大语言模型能否作为空间感知的零样本异常检测器，通过将视频转换为文本描述并利用文本蕴含进行标签打分。我们在UCF-Crime和RWF-2000数据集上测试了四个开源模型，在提示词与隐私保护条件下进行评估。少量示例可提升部分模型准确率，但可能增加误报；隐私过滤器——尤其是全身GAN变换——引入不一致性，降低准确率。结果揭示当前VLMs在简单、空间显著事件上有效，但在噪声空间线索或身份模糊情况下表现不佳。未来方向包括结构感知提示、跨片段轻量级空间记忆、描述阶段引入场景图或3D姿态先验，以及保留动作相关几何信息的隐私方法。这使零样本、语言引导的流水线成为具身化真实世界视频理解的可扩展基础组件。代码已公开：https://github.com/pascalbenschopTU/VLLM_AnomalyRecognition
原文摘要 · Abstract (English)
The widespread use of cameras in our society has created an overwhelming amount of video data, far exceeding the capacity for human monitoring. This presents a critical challenge for public safety and security, as the timely detection of anomalous or criminal events is crucial for effective response and prevention. The ability for an embodied agent to recognize unexpected events is fundamentally tied to its capacity for spatial reasoning. This paper investigates the spatial reasoning of vision-language models (VLMs) by framing anomalous action recognition as a zero-shot, language-grounded task, addressing the embodied perception challenge of interpreting dynamic 3D scenes from sparse 2D video. Specifically, we investigate whether small, pre-trained vision--LLMs can act as spatially-grounded, zero-shot anomaly detectors by converting video into text descriptions and scoring labels via textual entailment. We evaluate four open models on UCF-Crime and RWF-2000 under prompting and privacy-preserving conditions. Few-shot exemplars can improve accuracy for some models, but may increase false positives, and privacy filters -- especially full-body GAN transforms -- introduce inconsistencies that degrade accuracy. These results chart where current vision--LLMs succeed (simple, spatially salient events) and where they falter (noisy spatial cues, identity obfuscation). Looking forward, we outline concrete paths to strengthen spatial grounding without task-specific training: structure-aware prompts, lightweight spatial memory across clips, scene-graph or 3D-pose priors during description, and privacy methods that preserve action-relevant geometry. This positions zero-shot, language-grounded pipelines as adaptable building blocks for embodied, real-world video understanding. Our implementation for evaluating VLMs is publicly available at: https://github.com/pascalbenschopTU/VLLM_AnomalyRecognition
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。