通过统一优化提示与表征，提升目标检测在无目标域图像下的零样本适应能力。
UPRE: Zero-Shot Domain Adaptation for Object Detection via Unified Prompt and Representation Enhancement
- 联合优化文本提示与视觉表征，解决检测任务与视觉语言模型不匹配问题。
- 在九个基准数据集上实现领先性能，显著优于现有零样本域适应方法。
- 适合关注跨域目标检测、视觉语言模型应用的研究者和工程师。
零样本域适应（ZSDA）因目标域缺乏图像而面临巨大挑战。现有方法利用视觉语言模型（VLMs）的零样本学习能力应对域分布偏移，但忽视了检测任务与VLM之间提示不匹配的问题。为此，我们提出统一提示与表征增强（UPRE）框架，联合优化文本提示与视觉表征。具体地，引入多视角域提示，融合语言域先验与检测特定知识；设计视觉表征增强模块，生成域风格变化。此外，采用多层次增强策略，包括相对域距离对齐与正负样本分离，分别在图像级对齐多模态表征、在实例级捕捉多样化视觉特征。在九个基准数据集上的大量实验表明，该框架在零样本域适应检测场景中表现优异。代码已公开于 https://github.com/AMAP-ML/UPRE。
原文摘要 · Abstract (English)
Zero-shot domain adaptation (ZSDA) presents substantial challenges due to the lack of images in the target domain. Previous approaches leverage Vision-Language Models (VLMs) to tackle this challenge, exploiting their zero-shot learning capabilities. However, these methods primarily address domain distribution shifts and overlook the misalignment between the detection task and VLMs, which rely on manually crafted prompts. To overcome these limitations, we propose the unified prompt and representation enhancement (UPRE) framework, which jointly optimizes both textual prompts and visual representations. Specifically, our approach introduces a multi-view domain prompt that combines linguistic domain priors with detection-specific knowledge, and a visual representation enhancement module that produces domain style variations. Furthermore, we introduce multi-level enhancement strategies, including relative domain distance and positive-negative separation, which align multi-modal representations at the image level and capture diverse visual representations at the instance level, respectively. Extensive experiments conducted on nine benchmark datasets demonstrate the superior performance of our framework in ZSDA detection scenarios. Code is available at https://github.com/AMAP-ML/UPRE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。