arXiv:2507.17394cs.CVcs.AI2025-07中稿 · ACM MM 2025被引 11

不微调大模型,用隐藏状态探测视频异常,效果更好且通用性强。

HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMs

  • 利用多模态大模型中间层隐藏状态,捕捉异常敏感特征。
  • 在UCF-Crime和XD-Violence上优于现有免训练方法,准确率提升显著。
  • 无需微调即可跨模型通用,适合快速部署的智能监控场景。

视频异常检测(VAD)旨在识别视频序列中偏离正常模式的异常行为。传统方法常面临高计算开销和对大量标注数据的依赖,限制了实际应用。为此,我们提出HiProbe-VAD,一种无需微调的新型框架,直接利用预训练多模态大语言模型(MLLMs)进行异常检测。本文发现,MLLMs的中间隐藏状态包含信息丰富的表示,相比输出层对异常具有更高敏感性和线性可分性。为此,我们设计动态层显著性探测(DLSP)机制,智能识别并提取推理过程中最具有信息量的中间层隐藏状态。随后,通过轻量级异常评分器与时间定位模块,基于这些状态高效检测异常并生成解释。在UCF-Crime和XD-Violence数据集上的实验表明,HiProbe-VAD优于现有免训练方法及多数传统方法。此外,该框架在不同预训练MLLMs间展现出卓越的跨模型泛化能力,无需任何调优,充分释放了预训练多模态大模型在视频异常检测中的潜力,为更实用、可扩展的解决方案铺平道路。

原文摘要 · Abstract (English)

Video Anomaly Detection (VAD) aims to identify and locate deviations from normal patterns in video sequences. Traditional methods often struggle with substantial computational demands and a reliance on extensive labeled datasets, thereby restricting their practical applicability. To address these constraints, we propose HiProbe-VAD, a novel framework that leverages pre-trained Multimodal Large Language Models (MLLMs) for VAD without requiring fine-tuning. In this paper, we discover that the intermediate hidden states of MLLMs contain information-rich representations, exhibiting higher sensitivity and linear separability for anomalies compared to the output layer. To capitalize on this, we propose a Dynamic Layer Saliency Probing (DLSP) mechanism that intelligently identifies and extracts the most informative hidden states from the optimal intermediate layer during the MLLMs reasoning. Then a lightweight anomaly scorer and temporal localization module efficiently detects anomalies using these extracted hidden states and finally generate explanations. Experiments on the UCF-Crime and XD-Violence datasets demonstrate that HiProbe-VAD outperforms existing training-free and most traditional approaches. Furthermore, our framework exhibits remarkable cross-model generalization capabilities in different MLLMs without any tuning, unlocking the potential of pre-trained MLLMs for video anomaly detection and paving the way for more practical and scalable solutions.

视频异常检测多模态大模型免微调隐藏状态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。