引入负向视觉提示,提升开放集目标检测抗干扰能力
T-Rex-Omni: Integrating Negative Visual Prompt in Generic Object Detection
- 用统一编码器同时处理正负视觉提示
- 零样本检测在LVIS数据集上达51.2 AP_r
- 适合需要抗混淆的开放集检测场景
目标检测方法已从封闭集演进至开放集范式。然而,现有开放集检测器仍仅依赖文本描述或视觉样例等正向提示,对视觉相似但语义不同的干扰项敏感。本文提出T-Rex-Omni框架,通过引入负向视觉提示来排除难分负样本。首先设计统一视觉提示编码器,联合处理正负提示;其次提出无需训练的负负计算(NNC)模块,在概率计算阶段动态抑制负响应;最后引入负负铰链损失(NNH),强化正负嵌入间的判别边界。该框架支持仅正向或正负联合推理模式,兼容用户指定或自动生成的负样本。大量实验表明,其在零样本检测中表现卓越,显著缩小了视觉提示与文本提示方法间的性能差距,并在长尾场景下表现突出(LVIS-minival上达51.2 AP_r)。本工作确立负向提示为推进开放集视觉识别的关键新维度。
原文摘要 · Abstract (English)
Object detection methods have evolved from closed-set to open-set paradigms over the years. Current open-set object detectors, however, remain constrained by their exclusive reliance on positive indicators based on given prompts like text descriptions or visual exemplars. This positive-only paradigm experiences consistent vulnerability to visually similar but semantically different distractors. We propose T-Rex-Omni, a novel framework that addresses this limitation by incorporating negative visual prompts to negate hard negative distractors. Specifically, we first introduce a unified visual prompt encoder that jointly processes positive and negative visual prompts. Next, a training-free Negating Negative Computing (NNC) module is proposed to dynamically suppress negative responses during the probability computing stage. To further boost performance through fine-tuning, our Negating Negative Hinge (NNH) loss enforces discriminative margins between positive and negative embeddings. T-Rex-Omni supports flexible deployment in both positive-only and joint positive-negative inference modes, accommodating either user-specified or automatically generated negative examples. Extensive experiments demonstrate remarkable zero-shot detection performance, significantly narrowing the performance gap between visual-prompted and text-prompted methods while showing particular strength in long-tailed scenarios (51.2 AP_r on LVIS-minival). This work establishes negative prompts as a crucial new dimension for advancing open-set visual recognition systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。