通过融合全局与局部特征,提升开放词汇目标检测对新类别的识别能力。
DetRefiner: Model-Agnostic Detection Refinement with Feature Fusion Transformer

- 用轻量Transformer融合图像级与区域级特征,动态校准检测置信度。
- 在COCO等数据集上,新类别检测准确率最高提升10.1 AP。
- 无需重训练主模型,可通用适配多种检测框架,适合实际部署。
开放词汇目标检测(OVOD)旨在识别已见与未见类别,但现有方法常因未能充分整合全局与局部上下文信息而难以泛化到新物体。本文提出DetRefiner,一种简单有效的即插即用框架,通过融合全局与局部特征来优化开放词汇检测结果。DetRefiner利用基础模型(如DINOv3)输出的全局图像特征和局部图像块特征,经轻量级Transformer编码器生成图像级属性向量与区域属性向量,并据此推断属性可靠性,重新校准基线模型的置信度。值得注意的是,DetRefiner可独立训练,无需访问基线模型内部特征,也无需重新训练。推理时仅依赖基线检测器的预测结果,输出辅助校准分数,与原始分数融合后得到最终置信度。尽管结构简洁,该方法在COCO、LVIS、ODinW13和Pascal VOC等多个数据集上持续提升多个OVOD模型性能,新类别检测平均提升达+10.1 AP。结果表明,学习融合全局与局部表示是一种强大且通用的开放世界目标检测推进机制。代码与模型已在GitHub开源。
原文摘要 · Abstract (English)
Open-vocabulary object detection (OVOD) aims to detect both seen and unseen categories, yet existing methods often struggle to generalize to novel objects due to limited integration of global and local contextual cues. We propose DetRefiner, a simple yet effective plug-and-play framework that learns to fuse global and local features to refine open-vocabulary detection. DetRefiner processes global image features and patch-level image features from foundational models (e.g., DINOv3) through a lightweight Transformer encoder. The encoder produces a class vector capturing image-level attributes and patch vectors representing local region attributes, from which attribute reliability is inferred to recalibrate the base model's confidence. Notably, DetRefiner is trained independently of the base OVOD model, requiring neither access to its internal features nor retraining. At inference, it operates solely on the base detector's predictions, producing auxiliary calibration scores that are merged with the base detector's scores to yield the final refined confidence. Despite this simplicity, DetRefiner consistently enhances multiple OVOD models across COCO, LVIS, ODinW13, and Pascal VOC, achieving gains of up to +10.1 AP on novel categories. These results highlight that learning to fuse global and local representations offers a powerful and general mechanism for advancing open-world object detection. Our codes and models are available at https://github.com/hitachi-rd-cv/detrefiner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。