让视频异常检测能精准定位并解释细微异常,像人一样看懂异常细节。
Text-guided Fine-Grained Video Anomaly Understanding
- 用视觉-文本对齐生成像素级异常热图,实现细粒度定位。
- 在ShanghaiTech和UBnormal上提升定位精度与判断准确率,支持可解释推理。
- 适合需要理解异常原因的场景,如安防监控、工业质检。
视频中细微异常常表现为微弱的时空线索,易被传统异常检测系统忽略。现有方法多提供粗粒度二分类结果,缺乏可解释性证据;而大视觉语言模型虽能生成文本判断,却无法精确定位细微视觉信号。为此,我们提出T-VAU框架,将细微异常证据融入多模态推理。具体地,设计异常热图解码器(AHD),通过视觉-文本特征对齐,从中间视觉表征中提取像素级时空异常热图;进一步设计区域感知异常编码器(RAE),将热图转换为结构化提示嵌入,使大视觉语言模型在统一推理流程中完成异常检测、定位与语义解释。为支持细粒度监督,构建基于ShanghaiTech和UBnormal的目标级细粒度视频-文本异常数据集,包含对象外观、定位及运动轨迹的详细标注。大量实验表明,T-VAU在两个基准上显著提升异常定位与文本推理性能,取得优异的BLEU-4指标与真/假判断准确率,同时提供可解释的像素级时空证据,助力异常理解。代码将公开于https://github.com/momiji-bit/T-VAU。
原文摘要 · Abstract (English)
Subtle abnormal events in videos often manifest as weak spatio-temporal cues that are easily overlooked by conventional anomaly detection systems. Existing video anomaly detection approaches typically provide coarse binary anomaly decisions without interpretable evidence, while large vision-language models (LVLMs) can produce textual judgments but lack precise localization of subtle visual signals. To address this gap, we propose Text-guided Fine-Grained Video Anomaly Understanding T-VAU, a framework that grounds subtle anomaly evidence into multimodal reasoning. Specifically, we introduce an Anomaly Heatmap Decoder (AHD) that performs visual-textual feature alignment to extract pixel-level spatio-temporal anomaly heatmaps from intermediate visual representations. We further design a Region-aware Anomaly Encoder (RAE) that converts these heatmaps into structured prompt embeddings, enabling the LVLM to perform anomaly detection, localization, and semantic explanation in a unified reasoning pipeline. To support fine-grained supervision, we construct a target-level fine-grained video-text anomaly dataset derived from ShanghaiTech and UBnormal with detailed annotations of object appearance, localization, and motion trajectories. Extensive experiments demonstrate that T-VAU significantly improves anomaly localization and textual reasoning performance on both benchmarks, achieving strong results in BLEU-4 metrics and Yes/No decision accuracy while providing interpretable pixel-level spatio-temporal evidence for anomaly understanding. The code will be available at https://github.com/momiji-bit/T-VAU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。