提升细粒度开放词汇检测中属性识别能力
DSAA: Dual-Stage Attribute Activation for Fine-grained Open Vocabulary Detection

- 分两阶段增强属性语义:文本嵌入与BERT编码时注入属性先验
- 在FG-OVD基准上显著提升未见属性类别的检测准确率
- 适合需要精细区分颜色、材质等属性的视觉识别任务
开放词汇目标检测(OVD)模型突破了封闭集检测的限制,可通过自然语言提示识别未见类别。然而,在涉及颜色、材质、纹理等属性的细粒度检测任务中表现不佳。我们发现其性能瓶颈源于类别信号过强时,属性信息被边缘化,导致属性与目标物体错误绑定。为此,提出双阶段属性激活(DSAA)框架:在文本嵌入阶段,使用属性前缀适配器(APA)生成显式属性先验;在BERT编码阶段,通过关键/值调制模块(K/V Modulator)选择性增强对应属性词的键值向量。此外,引入属性感知对比损失,提升同类别但属性不同的实例间的区分度。在FG-OVD基准上的实验表明,该方法在多种主流OVD模型上均有效提升细粒度检测性能。
原文摘要 · Abstract (English)
Open-Vocabulary Object Detection (OVD) models break the limitations of closed-set detection, enabling the identification of unseen categories through natural language prompts. However, they exhibit notable limitations in fine-grained detection tasks involving attributes like color, material, and texture. We attribute this performance bottleneck in OVD models to a core issue: when category signals dominate, OVD models tend to marginalize attribute information during inference. This leads to incorrect binding between attributes and target objects. To address this, we propose the Dual-Stage Attribute Activation (DSAA) framework, which enhances fine-grained detection capabilities by strengthening attribute semantics at two critical stages. In the text embedding stage, we employ Attribute Prefix Adapter (APA) module to generate attribute prefixes that inject explicit attribute priors. To further amplify the influence of these attributes, our Key/Value (K/V) Modulator module then intervenes during the BERT encoding phase, selectively enhancing the Key and Value vectors of the corresponding attribute tokens. In addition, we introduce an attribute-aware contrastive loss to improve discrimination among same-category instances with different attributes during training. Experimental results on the FG-OVD benchmark demonstrate the effectiveness of our method across various mainstream open-vocabulary models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。