让检测模型持续学习新概念,不丢旧知识且无需重训。
COVD: Continual Open-Vocabulary Object Detection with Novel Concept Injection

- 冻结视觉编码器,只更新少量文本分支参数注入新概念
- 在Novel-114上实现新旧概念双优表现,性能优于现有方法
- 适合需要持续迭代的现实场景,如智能监控、自动驾驶
开放词汇目标检测(OVD)已取得显著进展,使检测器能从已见类别泛化到未见类别。然而,真实世界中的类别空间持续演化,现有OVD模型仍难以应对新出现的概念,而重复全量重训代价过高。为此,我们提出新任务设定——持续开放词汇检测与新概念注入(COVD),模型需依次学习新概念组,同时保留已有概念和原始开放词汇知识,并构建新基准Novel-114。关键观察是:预训练视觉编码器通常已感知并表征大量新概念,主要瓶颈在于视觉表示与文本概念间缺乏稳定语义对齐。基于此,我们提出无额外参数的高效注入框架NoIn-Det。该方法冻结视觉编码器,仅使用常见概念及先前注入概念的文本表示来保持文本空间,通过更新少量对新概念学习有益的文本分支参数实现新概念注入。大量实验表明,NoIn-Det能有效学习新概念,保留旧知识,且始终优于现有模型,无需引入额外参数。Novel-114与代码将公开。
原文摘要 · Abstract (English)
Open-vocabulary object detection (OVD) has made significant progress, enabling detectors to generalize from seen to unseen categories. However, real-world category spaces continually evolve, and existing OVD models still struggle with newly emerging concepts, while repeated full retraining is prohibitively expensive. To this end, we introduce a new task setting, termed Continual OVD with Novel Concept Injection (COVD), where models sequentially learn incoming novel concept groups while preserving prior concepts and original open-vocabulary knowledge, along with a new benchmark, Novel-114. Our key observation is that pretrained visual encoders often already perceive and represent many novel concepts, and the main bottleneck lies in the lack of stable semantic alignment between visual representations and textual concepts. Based on this, we propose NoIn-Det, an efficient continual injection framework without additional parameters. NoIn-Det freezes the visual encoder, preserves the text representation space using only texts of common concepts and previously injected concepts, and injects novel concepts by updating only a small subset of text-branch parameters beneficial to novel concept learning. Extensive experiments show that NoIn-Det effectively learns novel concepts, preserves old knowledge, and consistently outperforms existing continual learning methods for VLMs without introducing additional parameters.Novel-114 and the code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。