arXiv:2607.23981cs.CVcs.AI2026-07

用语言先验校准视觉检测,提升开放世界目标识别的准确率

Multimodal Semantic-Probabilistic Objectness for Open World Object Detection

论文配图:Multimodal Semantic-Probabilistic Objectness for Open World Object Detection
图 1 · 摘自论文原文
  • 引入语义校准框架,将已知类的语言描述融合到检测过程
  • 在M-OWODB和S-OWODB上提升2.7点mAP,未知召回率保持竞争力
  • 无需未来类别名称,适合真实开放环境下的增量学习场景

开放世界目标检测(OWOD)要求检测器识别已知类别、发现未命名的新类别对象,并增量学习新标注类别。现有方法PROB通过解码器查询空间中的类无关概率性物体性提升未知对象发现能力,但仅依赖视觉特征难以区分已知实例、未知对象与背景噪声,导致已知-未知边界模糊。本文提出轻量级语义校准框架MSPO,为PROB注入任务感知的已知类别语言先验,同时保持原检测架构与增量学习协议。针对每个已知类别,构建包含属性、外观、典型场景和功能用途的扩展文本描述,使用冻结的CLIP文本编码器编码;将解码器查询特征投影至同一语义空间,估计其对已知类语义的支持度。该语义证据与PROB的视觉物体性融合,校准已知与未知预测,避免将OWOD转为开集分类。关键在于,MSPO不使用未来类别名称,所有未见类别在评估时仍保持未命名。在M-OWODB与S-OWODB上的实验表明,MSPO在主流聚合指标上超越强基线PROB,同时保持竞争性未知召回率,还显著改善早期未知混淆指标,并使PASCAL VOC最终mAP最高提升2.7点。结果证明,在标准OWOD设置下,已知类语言语义可作为概率物体性的有效校准信号。

原文摘要 · Abstract (English)

Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incrementally learn newly annotated classes. PROB improves unknown discovery by modeling class-agnostic probabilistic objectness in the decoder-query space. However, visual objectness alone cannot determine whether an object-like query corresponds to a hard known instance, an unseen-category object, or background clutter, resulting in an ambiguous known-unknown decision boundary. We propose MSPO, a lightweight semantic calibration framework that augments PROB with task-aware known-category language priors while preserving its detector architecture and incremental learning protocol. For each currently known category, MSPO constructs an extended text description covering category attributes, visual appearance, typical scenes, and functional usage, and encodes it using a frozen CLIP text encoder. Decoder query features are projected into the same semantic space to estimate their support from the current known-category semantics. This semantic evidence is fused with PROB's visual objectness to calibrate known and unknown predictions without turning OWOD into open-vocabulary classification. Importantly, MSPO never uses future-category names, and all unseen categories remain unnamed during evaluation. Experiments on M-OWODB and S-OWODB show that MSPO improves the strong PROB baseline on the main aggregate metrics while retaining competitive unknown recall. It also improves early unknown-confusion metrics and raises PASCAL VOC final mAP by up to 2.7 points. These results demonstrate that known-category language semantics provide an effective calibration signal for probabilistic objectness under the standard OWOD setting.

开放世界检测多模态融合语言先验增量学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。