实测发现，主流异常检测模型在跨场景部署时效果几乎随机，远低于宣传水平。
Benchmark AUC Is Not Deployable Reliability: A Cross-Dataset Audit of Off-the-Shelf Features for Surveillance Video Anomaly Detection
- 用预训练模型提取特征，通过最近邻距离判断视频帧是否异常
- 同场景检测平均AUC为0.704，跨场景降至0.499（接近随机猜测）
- 即使优化参数，每小时误报仍超3万次，不适合实际部署
自动化可疑行为识别是智能监控的承诺之一，当前研究在标准数据集上报告了较高的帧级ROC-AUC。这些结果基于在同一摄像头和场景上训练与测试。我们检验当此假设被打破时的情况：从一个数据集的所有正常帧构建无监督正常性模型，使用冻结的现成嵌入（CLIP、DINOv2、ResNet-50、EfficientNet-B0）和最近邻距离，评估同一及其它数据集的测试帧。在4个真实数据集（UCSD Ped1、UCSD Ped2、CUHK Avenue、ShanghaiTech）和4种骨干网络下，同场景AUC平均为0.704，跨场景平均为0.499，处于随机水平；部分组合甚至低于随机。最强骨干模型（DINOv2）表现更差：同场景最高达0.901（Ped2），但跨场景下降最显著。该现象非评分规则所致——改用PaDiM风格的马氏距离检测器后，跨场景差距仍为0.202（原为0.208）。即便在理想工作点，误报率也高达每小时约31,931次。结论：现有基准指标反映的是受控实验室条件，严重夸大了实际部署可靠性，代码已公开。
原文摘要 · Abstract (English)
Automated "suspicious behavior" flagging is a headline promise of AI surveillance, and the field reports high frame-level ROC-AUC on standard video anomaly detection benchmarks. Those numbers are measured by training and testing on the same camera and scene. We audit what happens when that assumption is dropped. We build an unsupervised normality model from the all-normal training frames of one dataset, using frozen off-the-shelf embeddings (CLIP, DINOv2, ResNet-50, EfficientNet-B0) and a nearest-neighbour distance, and score the test frames of the same and of other datasets. Across 4 real datasets (UCSD Ped1, UCSD Ped2, CUHK Avenue, ShanghaiTech) and 4 backbones, same-dataset AUC averages 0.704 but cross-dataset AUC averages 0.499, which is chance: a detector calibrated on one scene is no better than a coin flip on another, and in several pairs it is below chance. The strongest backbone makes this worse, not better: DINOv2 has the best same-dataset AUC (up to 0.901 on Ped2) and the largest cross-dataset drop. The collapse is not an artefact of the scoring rule: replacing the nearest-neighbour detector with a PaDiM-style Mahalanobis detector reproduces it almost exactly (cross-dataset gap 0.202 versus 0.208). Even at a favourable operating point the false-alarm rate is on the order of 31,931 per hour. We conclude that the benchmark numbers quoted for surveillance anomaly detection describe a calibrated laboratory setting and overstate deployable reliability by a wide margin, and we release the code that reproduces every number.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。