用混合模型检测视频中复杂异常事件,提升准确率并减少计算开销。
HyCoVAD: A Hybrid SSL-LLM Model for Complex Video Anomaly Detection
- 结合自监督学习与大模型,分步处理视频帧与语义推理。
- 在ComplexVAD数据集上达72.5%帧级AUC,比基线高12.5%。
- 适合关注复杂场景异常检测的研究者与安防系统开发者。
视频异常检测对智能监控至关重要,但复杂异常(由多实体间复杂关系与时间依赖定义)难以识别。自监督学习(SSL)擅长建模低层时空模式,却难理解交互语义;大语言模型(LLM)具备强大上下文推理能力,但逐帧分析成本高且缺乏空间定位精度。本文提出HyCoVAD,一种融合SSL与LLM的混合模型:先用基于nnFormer的多任务SSL模块分析视频帧,筛选可疑片段;再将这些片段输入LLM,通过结构化规则推理验证异常存在。在挑战性ComplexVAD数据集上的实验表明,该模型实现72.5%的帧级AUC,优于现有基线12.5%,同时显著降低LLM计算量。我们公开了交互异常分类体系、自适应阈值协议及代码,以推动复杂视频异常检测研究。
原文摘要 · Abstract (English)
Video anomaly detection (VAD) is crucial for intelligent surveillance, but a significant challenge lies in identifying complex anomalies, which are events defined by intricate relationships and temporal dependencies among multiple entities rather than by isolated actions. While self-supervised learning (SSL) methods effectively model low-level spatiotemporal patterns, they often struggle to grasp the semantic meaning of these interactions. Conversely, large language models (LLMs) offer powerful contextual reasoning but are computationally expensive for frame-by-frame analysis and lack fine-grained spatial localization. We introduce HyCoVAD, Hybrid Complex Video Anomaly Detection, a hybrid SSL-LLM model that combines a multi-task SSL temporal analyzer with LLM validator. The SSL module is built upon an nnFormer backbone which is a transformer-based model for image segmentation. It is trained with multiple proxy tasks, learns from video frames to identify those suspected of anomaly. The selected frames are then forwarded to the LLM, which enriches the analysis with semantic context by applying structured, rule-based reasoning to validate the presence of anomalies. Experiments on the challenging ComplexVAD dataset show that HyCoVAD achieves a 72.5% frame-level AUC, outperforming existing baselines by 12.5% while reducing LLM computation. We release our interaction anomaly taxonomy, adaptive thresholding protocol, and code to facilitate future research in complex VAD scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。