无需额外监督，利用视觉知识提升无人机目标检测的开放词汇能力
VK-Det: Visual Knowledge Guided Prototype Learning for Open-Vocabulary Aerial Object Detection
- 用视觉特征自发发现关键区域，实现细粒度定位与自适应蒸馏
- 通过原型匹配将检测区域映射到潜在类别，提升对新类别的识别
- 在无文本依赖情况下仍达顶尖性能，适合开放场景目标检测
为识别预定义类别之外的目标，开放词汇航空目标检测（OVAD）利用视觉语言模型（VLM）的零样本能力，从基础类别泛化到新类别。现有方法通常依赖弱文本监督进行自学习，生成区域级伪标签以对齐检测器与VLM语义空间。但文本依赖导致语义偏差，限制了开放词汇扩展范围。我们提出VK-Det，一种无需额外监督的视觉知识引导式开放词汇目标检测框架。首先，发现并利用视觉编码器固有的信息区域感知能力，实现细粒度定位与自适应蒸馏；其次，引入新型原型感知伪标签策略，通过特征聚类建模类别间决策边界，并通过原型匹配将检测区域映射至潜在类别，增强对新类别的关注并弥补监督缺失。大量实验表明，该方法达到领先性能，在DIOR上获得30.1 mAP^N，DOTA上达23.3 mAP^N，超越部分需额外监督的方法。
原文摘要 · Abstract (English)
To identify objects beyond predefined categories, open-vocabulary aerial object detection (OVAD) leverages the zero-shot capabilities of visual-language models (VLMs) to generalize from base to novel categories. Existing approaches typically utilize self-learning mechanisms with weak text supervision to generate region-level pseudo-labels to align detectors with VLMs semantic spaces. However, text dependence induces semantic bias, restricting open-vocabulary expansion to text-specified concepts. We propose $\textbf{VK-Det}$, a $\textbf{V}$isual $\textbf{K}$nowledge-guided open-vocabulary object $\textbf{Det}$ection framework $\textit{without}$ extra supervision. First, we discover and leverage vision encoder's inherent informative region perception to attain fine-grained localization and adaptive distillation. Second, we introduce a novel prototype-aware pseudo-labeling strategy. It models inter-class decision boundaries through feature clustering and maps detection regions to latent categories via prototype matching. This enhances attention to novel objects while compensating for missing supervision. Extensive experiments show state-of-the-art performance, achieving 30.1 $\mathrm{mAP}^{N}$ on DIOR and 23.3 $\mathrm{mAP}^{N}$ on DOTA, outperforming even extra supervised methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。