不微调大模型,直接找关键注意力头实现高效异常检测
HeadHunt-VAD: Hunting Robust Anomaly-Sensitive Heads in MLLM for Tuning-Free Video Anomaly Detection
- 从冻结的多模态大模型中挖掘对异常敏感的注意力头
- 在两个基准上达到当前无微调方法最佳性能,效率高
- 适合需要快速部署、解释性强的视频异常检测场景
视频异常检测(VAD)旨在定位偏离正常模式的事件。传统方法依赖大量标注数据且计算成本高。近期基于多模态大语言模型(MLLM)的无微调方法利用其丰富世界知识提供了新思路,但通常依赖文本输出,存在信息丢失、正常性偏差和提示敏感等问题,难以捕捉细微异常线索。为此,我们提出HeadHunt-VAD,一种新型无微调VAD范式,通过直接探测冻结MLLM内部的注意力头来规避文本生成。核心是鲁棒头识别模块,采用多标准分析(显著性与稳定性)系统评估所有注意力头,识别出在多种提示下均具判别力的稀疏头集。这些专家头的特征输入轻量级异常评分器与时间定位器,实现高效准确的异常检测并提供可解释输出。大量实验表明,HeadHunt-VAD在两个主流VAD基准上超越现有无微调方法,同时保持高效率,验证了MLLM头级探查作为实际可行异常检测方案的有效性。
原文摘要 · Abstract (English)
Video Anomaly Detection (VAD) aims to locate events that deviate from normal patterns in videos. Traditional approaches often rely on extensive labeled data and incur high computational costs. Recent tuning-free methods based on Multimodal Large Language Models (MLLMs) offer a promising alternative by leveraging their rich world knowledge. However, these methods typically rely on textual outputs, which introduces information loss, exhibits normalcy bias, and suffers from prompt sensitivity, making them insufficient for capturing subtle anomalous cues. To address these constraints, we propose HeadHunt-VAD, a novel tuning-free VAD paradigm that bypasses textual generation by directly hunting robust anomaly-sensitive internal attention heads within the frozen MLLM. Central to our method is a Robust Head Identification module that systematically evaluates all attention heads using a multi-criteria analysis of saliency and stability, identifying a sparse subset of heads that are consistently discriminative across diverse prompts. Features from these expert heads are then fed into a lightweight anomaly scorer and a temporal locator, enabling efficient and accurate anomaly detection with interpretable outputs. Extensive experiments show that HeadHunt-VAD achieves state-of-the-art performance among tuning-free methods on two major VAD benchmarks while maintaining high efficiency, validating head-level probing in MLLMs as a powerful and practical solution for real-world anomaly detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。