用大模型理解视频异常的因果关系,提升解释能力。
VADER: Towards Causal Video Anomaly Understanding with Relation-Aware Large Language Models
- 结合关键帧物体关系与视觉线索,建模动态交互
- 在多个真实数据集上实现异常描述与因果推理领先性能
- 适合需要可解释视频分析的场景,如安防监控
视频异常理解(VAU)旨在对视频中的异常事件提供详细解释与语义理解,克服传统方法仅关注检测与定位的局限。现有方法常忽略物体间的深层因果关系与交互,而这些对理解异常行为至关重要。本文提出VADER,一种基于大语言模型的视频异常理解框架,将关键帧物体关系特征与视觉线索融合,增强异常理解。首先通过异常评分器为每帧分配异常分数,再采用上下文感知采样(CAES)策略捕捉异常事件的因果背景。关系特征提取器与对比关系编码器(CORE)协同建模动态物体交互,生成紧凑的关系表征用于下游推理。视觉与关系线索与大模型结合,生成具因果依据的详细描述,并支持鲁棒的异常相关问答。在多个真实世界VAU基准测试中,VADER在异常描述、解释与因果推理任务上均取得优异表现,推动可解释视频异常分析的前沿进展。
原文摘要 · Abstract (English)
Video anomaly understanding (VAU) aims to provide detailed interpretation and semantic comprehension of anomalous events within videos, addressing limitations of traditional methods that focus solely on detecting and localizing anomalies. However, existing approaches often neglect the deeper causal relationships and interactions between objects, which are critical for understanding anomalous behaviors. In this paper, we propose VADER, an LLM-driven framework for Video Anomaly unDErstanding, which integrates keyframe object Relation features with visual cues to enhance anomaly comprehension from video. Specifically, VADER first applies an Anomaly Scorer to assign per-frame anomaly scores, followed by a Context-AwarE Sampling (CAES) strategy to capture the causal context of each anomalous event. A Relation Feature Extractor and a COntrastive Relation Encoder (CORE) jointly model dynamic object interactions, producing compact relational representations for downstream reasoning. These visual and relational cues are integrated with LLMs to generate detailed, causally grounded descriptions and support robust anomaly-related question answering. Experiments on multiple real-world VAU benchmarks demonstrate that VADER achieves strong results across anomaly description, explanation, and causal reasoning tasks, advancing the frontier of explainable video anomaly analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。