让检测器同时识别新类别和新场景,且不需重新训练。
ExDet: Open-Domain Open-Vocabulary Detection with Cross-modal Extrapolation and Rectification

- 用文本生成虚拟视觉原型,指导模型跨类别跨域泛化。
- 推理时轻量修正特征分布,提升新类别与未知场景识别率。
- 无需重训模型,适合快速部署到新任务或新数据域。
开放域开放词汇检测(ODOVD)要求检测器同时适应新类别和未知数据域,挑战性高于传统开放词汇检测。现有方法通常从头训练检测器与域泛化模块,成本高昂。本文提出ExDet,一种轻量级类别-域协同泛化框架,可增强已有检测器的跨类别与跨域泛化能力。ExDet包含文本引导外推(TGE)、检测器无关修正(DCR)模块和ExRPN。TGE利用视觉语言模型(VLM)的DeltaSpace特性,从文本中推断出类别与域感知的虚拟视觉原型。DCR在无需检测器训练和真实数据的情况下,基于TGE生成的原型学习,插入分类头后进行推理时修正特征表示,使其向源域视觉分布靠拢,从而提升对新类别和未见域目标的分类性能。ExRPN通过结合语义相似性与RPN置信度重新校准候选框得分,提高对新类别及域偏移物体的召回率,并为后续分类与DCR提供更好支持。ExDet在OD-LVIS、OV-LVIS、Objects365和MSOSB上均达到当前最优性能。
原文摘要 · Abstract (English)
Open-domain open-vocabulary detection (ODOVD) requires detectors to generalize to both novel categories and unseen domains, making it more challenging than open-vocabulary detection. Existing methods typically train open-vocabulary detectors together with domain generalization modules from scratch, leading to high training cost. we propose ExDet, a lightweight category-domain collaborative generalization framework for ODOVD that enhances the cross-category and cross-domain generalization of existing detectors. ExDet consists of Text-Guided Extrapolation (TGE), a lightweight Detector-Compatible Rectification (DCR) module, and ExRPN. Specifically, TGE exploits the DeltaSpace property of vision-language models (VLMs) to infer category- and domain-aware proxy visual prototypes from text. DCR is learned from the TGE-generated prototypes in a detector training-free and real-data-free manner, and is inserted after the classification head at inference to rectify representations toward a detector-compatible source-domain visual distribution, thereby enhancing classification for targets from novel categories and unseen domains. ExRPN recalibrates proposal scores by combining semantic similarity with RPN confidence, improving recall for novel and domain-shifted objects while providing better support for subsequent classification and DCR. ExDet achieves SOTA performance on OD-LVIS, OV-LVIS, Objects365, and MSOSB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。