解决细粒度开放词汇检测中属性与主体混淆问题,提升识别精度
GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection
- 将定位与细粒度识别分离,用主体嵌入稳定定位
- 通过注意力机制融合关键属性,避免属性过度主导
- 适合需要精准识别细分类别场景的视觉系统
细粒度开放词汇目标检测(FG-OVD)旨在检测由属性丰富文本描述的新物体类别。现有开放词汇检测器在基础类别上表现良好,但在细粒度设置下因预训练视觉语言模型(VLM)嵌入中主体与属性语义纠缠,导致属性过表示、定位错误和嵌入空间语义漂移。本文提出GUIDED,一种专为解耦细粒度提示中主体与属性语义而设计的分解框架。给定细粒度类别名,先用语言模型提取粗粒度主体及其描述属性;随后仅用主体嵌入引导检测器进行定位,确保不受无关属性干扰。引入基于注意力的属性嵌入融合模块,选择性保留有助于区分的属性信息。最后,区域级属性判别模块利用改进的视觉语言模型与投影头,对比检测区域与完整类别名,增强对齐效果。在FG-OVD和3F-OVD基准上的大量实验表明,GUIDED达到新最优性能,验证了解耦建模与模块优化的有效性。代码将公开于https://github.com/lijm48/GUIDED。
原文摘要 · Abstract (English)
Fine-grained open-vocabulary object detection (FG-OVD) aims to detect novel object categories described by attribute-rich texts. While existing open-vocabulary detectors show promise at the base-category level, they underperform in fine-grained settings due to the semantic entanglement of subjects and attributes in pretrained vision-language model (VLM) embeddings -- leading to over-representation of attributes, mislocalization, and semantic drift in embedding space. We propose GUIDED, a decomposition framework specifically designed to address the semantic entanglement between subjects and attributes in fine-grained prompts. By separating object localization and fine-grained recognition into distinct pathways, HUIDED aligns each subtask with the module best suited for its respective roles. Specifically, given a fine-grained class name, we first use a language model to extract a coarse-grained subject and its descriptive attributes. Then the detector is guided solely by the subject embedding, ensuring stable localization unaffected by irrelevant or overrepresented attributes. To selectively retain helpful attributes, we introduce an attribute embedding fusion module that incorporates attribute information into detection queries in an attention-based manner. This mitigates over-representation while preserving discriminative power. Finally, a region-level attribute discrimination module compares each detected region against full fine-grained class names using a refined vision-language model with a projection head for improved alignment. Extensive experiments on FG-OVD and 3F-OVD benchmarks show that GUIDED achieves new state-of-the-art results, demonstrating the benefits of disentangled modeling and modular optimization. Our code will be released at https://github.com/lijm48/GUIDED.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。