arXiv:2509.04833cs.CVcs.AI2025-09ICCV被引 12

提出首个端到端提案驱动的视觉定位框架,提升复杂场景下的目标识别能力。

PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discrimination

  • 通过端到端架构融合目标提案生成与指代表达理解,无需额外检测器。
  • 在gRefCOCO等4个基准上实现最新最佳性能,最高提升2.3个点。
  • 引入多粒度区分机制,有效处理缺失目标和细粒度语义差异。

近年来,视觉定位研究逐渐摒弃传统两阶段提案方法,转向端到端直接引用范式。然而,现有方法仅依赖被指代目标进行监督,忽略了显著前景目标的潜在价值。同时,多数方法缺乏多粒度区分能力,难以应对复杂场景中的鲁棒目标识别。为此,我们提出PropVG,一个端到端提案驱动的框架,据我们所知是首个无缝整合前景目标提案生成与指代表达理解而无需额外检测器的方法。此外,我们设计了基于对比学习的参考评分(CRS)模块,在句子和词级别进行对比学习,增强对被指代对象的理解与区分能力。还提出了多粒度目标区分(MTD)模块,融合对象级与语义级信息,提升对缺失目标的识别能力。在gRefCOCO(GREC/GRES)、Ref-ZOM、R-RefCOCO和RefCOCO(REC/RES)等多个基准上的大量实验表明,PropVG具有显著有效性。代码与模型已开源:https://github.com/Dmmm1997/PropVG。

原文摘要 · Abstract (English)

Recent advances in visual grounding have largely shifted away from traditional proposal-based two-stage frameworks due to their inefficiency and high computational complexity, favoring end-to-end direct reference paradigms. However, these methods rely exclusively on the referred target for supervision, overlooking the potential benefits of prominent prospective targets. Moreover, existing approaches often fail to incorporate multi-granularity discrimination, which is crucial for robust object identification in complex scenarios. To address these limitations, we propose PropVG, an end-to-end proposal-based framework that, to the best of our knowledge, is the first to seamlessly integrate foreground object proposal generation with referential object comprehension without requiring additional detectors. Furthermore, we introduce a Contrastive-based Refer Scoring (CRS) module, which employs contrastive learning at both sentence and word levels to enhance the capability in understanding and distinguishing referred objects. Additionally, we design a Multi-granularity Target Discrimination (MTD) module that fuses object- and semantic-level information to improve the recognition of absent targets. Extensive experiments on gRefCOCO (GREC/GRES), Ref-ZOM, R-RefCOCO, and RefCOCO (REC/RES) benchmarks demonstrate the effectiveness of PropVG. The codes and models are available at https://github.com/Dmmm1997/PropVG.

视觉定位端到端多粒度对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。