arXiv:2607.11008cs.CVcs.AI2026-07中稿 · CVPR

解决图像文本对齐中同义词导致定位不一致的问题。

SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception

论文配图:SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
图 1 · 摘自论文原文
  • 引入语义一致的空间注意力对齐模块,减少同义表达的注意力差异。
  • 在多个基准上实现同义词场景下定位一致性显著提升,性能领先现有方法。
  • 适合需要鲁棒开放词汇感知的视觉理解任务,如复杂场景目标检测。

开放词汇密集感知(OVDP)旨在通过文本知识定位训练中未见的物体。尽管基于CLIP的方法取得显著进展,我们发现一个关键问题:同义词引发的定位不一致,即语义等价的表达产生不同的空间注意力模式,削弱了真实应用中的鲁棒性与性能。为此,提出SynCLIP,一种同义词一致的语言-图像预训练框架,以增强OVDP的同义词鲁棒性。SynCLIP引入语义一致的空间注意力对齐(SSA)模块,通过最小化原表达与同义表达间的注意力图差异来提升一致性;同时设计空间注意力优化(SAR)模块,有选择地强化对齐图中最相关的空间区域,实现更精确稳定的定位。为支持同义词一致预训练,构建了同义词丰富视觉语料库(SEViC),每个类别均扩展多个同义词和文本定义。大量实验表明,SynCLIP在多种语言变体下显著提升定位一致性,并在基于CLIP的OVDP方法中达到最先进水平。

原文摘要 · Abstract (English)

Open-vocabulary dense perception (OVDP) aims to localize objects unseen during training by leveraging textual knowledge. Despite the remarkable progress of recent CLIP-based approaches, we identify a critical limitation: synonym-induced grounding inconsistency, where semantically equivalent expressions yield disparate spatial attention patterns. This inconsistency undermines the robustness and performance of existing methods in real-world OVDP applications. To address this issue, we propose SynCLIP, a Synonym-Coherent Language-Image Pretraining framework that enhances synonym-robust grounding for OVDP. SynCLIP introduces a Semantic-consistent Spatial Attention alignment (SSA) module to enhance spatial attention consistency by minimizing discrepancies between attention maps of original and synonymous expressions. Furthermore, a Spatial Attention Refinement (SAR) module selectively strengthens the most semantically relevant spatial regions within aligned maps for more precise and stable grounding. To support synonym-coherent pretraining, we also construct a Synonym-Enriched Visual Corpus (SEViC), which augments each category with multiple synonyms and textual definitions. Extensive experiments on multiple benchmarks demonstrate that SynCLIP substantially improves grounding consistency under diverse linguistic variants and achieves state-of-the-art performance among CLIP-based OVDP methods. Code is available at https://github.com/Justlovesmile/SynCLIP.

开放词汇图像定位同义词对齐预训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。