用视觉大模型提升无源目标检测的泛化与判别能力
Beyond Boundaries: Leveraging Vision Foundation Models for Source-Free Object Detection
- 引入视觉大模型作为外部知识,改进特征对齐和标签质量
- 在六大数据集上达到当前最优性能,显著提升检测准确率
- 适合关注跨域检测与模型泛化能力的研究者
无源目标检测(SFOD)旨在不访问源数据的情况下,将预训练于源域的检测器适配到目标域。现有方法主要依赖源模型内部知识,限制了跨域泛化能力,并常导致偏差伪标签,影响迁移性和判别性。相比之下,视觉基础模型(VFMs)在海量多样数据上预训练,具备强大感知能力和广泛泛化性,但在SFOD中尚未被充分利用。本文提出一种新框架,将VFMs作为外部知识源,联合提升特征对齐与标签质量。设计三个基于VFMs的模块:(1) 基于补丁相似性的全局特征对齐(PGFA),通过加权提取全局特征以增强可迁移性;(2) 基于原型的实例级特征对齐(PIFA),利用动量更新的VFMs原型进行实例对比学习;(3) 双源增强伪标签融合(DEPF),通过熵感知策略融合检测类VFMs与教师模型预测,生成更可靠的监督信号。在六个基准上的大量实验表明,该方法在多种设置下均达到当前最优性能,验证了融合VFMs在同时提升迁移性与判别性方面的有效性。
原文摘要 · Abstract (English)
Source-Free Object Detection (SFOD) aims to adapt a source-pretrained object detector to a target domain without access to source data. However, existing SFOD methods predominantly rely on internal knowledge from the source model, which limits their capacity to generalize across domains and often results in biased pseudo-labels, thereby hindering both transferability and discriminability. In contrast, Vision Foundation Models (VFMs), pretrained on massive and diverse data, exhibit strong perception capabilities and broad generalization, yet their potential remains largely untapped in the SFOD setting. In this paper, we propose a novel SFOD framework that leverages VFMs as external knowledge sources to jointly enhance feature alignment and label quality. Specifically, we design three VFM-based modules: (1) Patch-weighted Global Feature Alignment (PGFA) distills global features from VFMs using patch-similarity-based weighting to enhance global feature transferability; (2) Prototype-based Instance Feature Alignment (PIFA) performs instance-level contrastive learning guided by momentum-updated VFM prototypes; and (3) Dual-source Enhanced Pseudo-label Fusion (DEPF) fuses predictions from detection VFMs and teacher models via an entropy-aware strategy to yield more reliable supervision. Extensive experiments on six benchmarks demonstrate that our method achieves state-of-the-art SFOD performance, validating the effectiveness of integrating VFMs to simultaneously improve transferability and discriminability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。