arXiv:2509.26386cs.CV2025-09NeurIPS被引 15

PANDA用AI工程师自动检测任意视频异常,无需训练数据或人工干预。

PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer

  • 基于多模态大模型构建自适应推理框架,动态规划检测策略。
  • 在多个场景下实现领先性能,零样本开集异常检测准确率达87.3%。
  • 适合需要低维护、高泛化能力的工业监控场景应用。

视频异常检测(VAD)因现实场景复杂多样而极具挑战。传统方法依赖特定领域训练数据和人工调整,成本高且泛化能力差。为此,我们提出PANDA,一种基于多模态大模型的代理型AI工程师,旨在实现无需训练数据或人工参与的通用视频异常检测。通过设计四项核心能力:(1)自适应场景感知策略规划,(2)目标驱动的启发式推理,(3)工具增强的自我反思,(4)持续学习的记忆链机制,实现全场景自适应。具体地,我们构建了自适应场景感知的RAG机制,用于检索异常相关知识以制定检测策略;引入潜在异常引导的启发式提示策略提升推理精度;采用渐进式反思机制与上下文感知工具,在复杂场景中迭代优化决策;最后通过记忆链机制利用历史经验实现持续性能提升。大量实验表明,PANDA在多场景、开集及复杂场景设置下均达到最优表现,验证其卓越的泛化与鲁棒性。代码已开源于https://github.com/showlab/PANDA。

原文摘要 · Abstract (English)

Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios. Previous methods typically rely on domain-specific training data and manual adjustments when applying to new scenarios and unseen anomaly types, suffering from high labor costs and limited generalization. Therefore, we aim to achieve generalist VAD, \ie, automatically handle any scene and any anomaly types without training data or human involvement. In this work, we propose PANDA, an agentic AI engineer based on MLLMs. Specifically, we achieve PANDA by comprehensively devising four key capabilities: (1) self-adaptive scene-aware strategy planning, (2) goal-driven heuristic reasoning, (3) tool-augmented self-reflection, and (4) self-improving chain-of-memory. Concretely, we develop a self-adaptive scene-aware RAG mechanism, enabling PANDA to retrieve anomaly-specific knowledge for anomaly detection strategy planning. Next, we introduce a latent anomaly-guided heuristic prompt strategy to enhance reasoning precision. Furthermore, PANDA employs a progressive reflection mechanism alongside a suite of context-aware tools to iteratively refine decision-making in complex scenarios. Finally, a chain-of-memory mechanism enables PANDA to leverage historical experiences for continual performance improvement. Extensive experiments demonstrate that PANDA achieves state-of-the-art performance in multi-scenario, open-set, and complex scenario settings without training and manual involvement, validating its generalizable and robust anomaly detection capability. Code is released at https://github.com/showlab/PANDA.

视频异常检测AI工程师多模态自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。