让检测器与大模型互学，提升弱监督视频异常检测效果
MuST-VAD: Mutual Structured Learning for Video Anomaly Detection
- 构建检测器与视觉语言模型双向知识交互机制
- 在UCF-Crime上将平均精度提升至42.46%，优于当前最优方法4.13点
- 适合关注弱监督异常检测与多模态协同学习的研究者
本文提出MuST-VAD，一种用于弱监督视频异常检测（VAD）的相互结构化学习框架。该框架使异常检测器与大型视觉语言模型（LVLM）之间实现双向知识交换。传统方法仅将LVLM语义单向迁移至检测器，而MuST-VAD通过循环更新：检测器预测结果用于指导LVLM适应，调整后的LVLM提供新表征重新训练检测器，两者在小视频组中交替优化。两模型均在检测器选中的关键片段上训练，并通过置信度加权和标注锚定问答确保监督可靠性。在UCF-Crime数据集上，该方法将单次传递基线的AUROC从88.15%提升至88.63%，平均精度（AP）从37.25%提升至42.46%，相较当前最佳方法提升4.13个百分点。
原文摘要 · Abstract (English)
In this paper, we propose MuST-VAD, a mutual structured learning framework for weakly supervised video anomaly detection (VAD) in which an anomaly detector and a large vision-language model (LVLM) exchange their acquired knowledge. Detectors in weakly supervised VAD learn anomaly scores from features extracted by a fixed, task-agnostic backbone. These fixed features bound the achievable detection accuracy. Recent methods therefore transfer LVLM semantics into the detector as richer features. However, this transfer is one-way: what the detector learns about the target videos never returns to the LVLM. MuST-VAD extends the one-way transfer into a bidirectional learning loop. In this loop, the latest detector predictions supervise the LVLM adaptation, and the adapted LVLM returns updated representations that retrain the detector; the two models alternate these updates over small video groups. Both models train on detector-selected key clips, while confidence weighting and annotation-anchored question answering keep the exchanged supervision reliable. On UCF-Crime, our mutual learning improves the one-pass transfer baseline from 88.15% to 88.63% AUROC and from 37.25% to 42.46% average precision (AP), outperforming the state-of-the-art method in AP by 4.13 points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。