arXiv:2605.03642cs.CV2026-05中稿 · ICPR 2026

让检测器自监督训练,提升视觉语言模型的局部细节识别能力。

The Detector Teaches Itself: Lightweight Self-Supervised Adaptation for Open-Vocabulary Object Detection

论文配图:The Detector Teaches Itself: Lightweight Self-Supervised Adaptation for Open-Vocabulary Object Detection
图 1 · 摘自论文原文
  • 用预训练检测器生成带伪标签的区域数据集,实现自监督微调。
  • 在COCO和LVIS上同时提升新旧类别检测性能,超越现有最佳水平。
  • 仅需微调0.8万参数,无推理开销,可直接嵌入现有系统。

开放词汇目标检测旨在识别开放类别中的物体,依赖于在大规模图像-文本数据上预训练的视觉语言模型(VLM)。协同范式将检测器与VLM结合,实现对新类别的零样本识别。然而,基于全图预训练的VLM往往难以捕捉局部物体细节,限制了其在区域级检测中的效果。本文提出解耦自适应训练(DAT),一种自监督微调方法,以增强VLM在协同模型中的表现。给定由封闭集检测器与VLM组成的协同模型,我们首先利用预训练的封闭集检测器构建一个区域感知的伪标签数据集,其中可能存在新类别物体但未被正确标注。随后,以解耦方式微调VLM的视觉主干,通过权重插值实现局部特征对齐,同时保留全局语义知识。DAT为即插即用模块,无需推理开销,且仅微调少于0.8M参数。在COCO和LVIS数据集上的实验表明,DAT在新类别与已知类别上均持续提升检测性能,建立协同开放词汇检测的新基准。

原文摘要 · Abstract (English)

Open-vocabulary object detection aims to recognize objects from an open set of categories, which leverages vision-language models (VLMs) pre-trained on large-scale image-text data. The cooperative paradigm combines an object detector with a VLM to achieve zero-shot recognition of novel objects. However, VLMs pre-trained on full images often struggle to capture local object details, limiting their effectiveness when applied to region-level detection. We present Decoupled Adaptivity Training (DAT), a self-supervised fine-tuning approach to improve VLMs for cooperative model-based object detection. Given a cooperative model consists of a closed-set detector and a VLM, we first construct a region-aware pseudo-labeled dataset using a pre-trained closed-set object detector, in which regions corresponding to novel objects may be present but remain unlabeled or mislabeled. We then fine-tune the visual backbone of the VLM in a decoupled manner, which enhances local feature alignment while preserving global semantic knowledge via weight interpolation. DAT is a plug-and-play module that requires no inference overhead and fine-tunes less than 0.8M parameters. Experiments on the COCO and LVIS datasets show that DAT consistently improves detection performance on both novel and known categories, establishing a new state of the art in cooperative open-vocabulary detection.

目标检测自监督视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。