arXiv:2502.05843cs.CV2025-02ICCV被引 3

用大模型引导符号推理,让目标检测器看懂复杂事件

From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors via LLM-guided Symbolic Reasoning

  • 通过符号回归与大模型引导,将物体检测结果转化为事件逻辑规则
  • 在非法捕捞等任务上提升8.36%至23.16%的识别准确率
  • 无需训练即可适配多种检测器,适合需要可解释事件理解的场景

当前目标检测器擅长实体定位与分类,但在事件识别方面存在固有局限,因其架构侧重离散物体识别,难以建模组合推理、物间关联与上下文语义。为此,我们提出一种新框架,通过大模型引导的符号推理,将标准检测器能力拓展至复杂事件理解,无需额外训练。该即插即用框架可对接任意开放词汇检测器,核心包含(i)探索检测实体间关系模式的符号回归机制,(ii)由大模型指导搜索有意义表达的战略性引导。所发现的符号规则将低层视觉感知转化为可解释的事件理解,提供跨领域强迁移性的透明推理路径。在多个应用领域对比专用事件识别系统,实验表明该框架使多种检测器在非法捕捞活动(75% AUROC,+8.36%)、施工安全违规(+15.77%)及异常人群行为(+23.16%)等任务中显著提升性能。代码已开源。

原文摘要 · Abstract (English)

Current object detectors excel at entity localization and classification, yet exhibit inherent limitations in event recognition capabilities. This deficiency arises from their architecture's emphasis on discrete object identification rather than modeling the compositional reasoning, inter-object correlations, and contextual semantics essential for comprehensive event understanding. To address this challenge, we present a novel framework that expands the capability of standard object detectors beyond mere object recognition to complex event understanding through LLM-guided symbolic reasoning. Our key innovation lies in bridging the semantic gap between object detection and event understanding without requiring expensive task-specific training. The proposed plug-and-play framework interfaces with any open-vocabulary detector while extending their inherent capabilities across architectures. At its core, our approach combines (i) a symbolic regression mechanism exploring relationship patterns among detected entities and (ii) a LLM-guided strategically guiding the search toward meaningful expressions. These discovered symbolic rules transform low-level visual perception into interpretable event understanding, providing a transparent reasoning path from objects to events with strong transferability across domains.We compared our training-free framework against specialized event recognition systems across diverse application domains. Experiments demonstrate that our framework enhances multiple object detector architectures to recognize complex events such as illegal fishing activities (75% AUROC, +8.36% improvement), construction safety violations (+15.77%), and abnormal crowd behaviors (+23.16%). Code is available at \href{https://github.com/MAC-AutoML/SymbolicDet}{here}.

事件理解符号推理大模型目标检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。