arXiv:2608.03991cs.CV2026-08

通过视觉原型校准文本嵌入,提升无训练开放词汇分割的准确性。

Perceptual Anchoring: Prototype-Guided Text Calibration for Training-free Open-Vocabulary Semantic Segmentation

论文配图:Perceptual Anchoring: Prototype-Guided Text Calibration for Training-free Open-Vocabulary Semantic Segmentation
图 1 · 摘自论文原文
  • 基于初始匹配得分选取可靠视觉证据,构建类别特定原型。
  • 用原型动态调整文本嵌入,使语义更贴合具体图像实例。
  • 无需训练或额外模型,可直接集成到现有方法中使用。

无训练开放词汇语义分割(OVSS)在不更新参数的情况下,依据任意文本描述对图像进行语义区域划分。然而,现有方法多聚焦于优化视觉表示,将仅编码通用类别概念的文本嵌入视为固定分类参考,导致通用概念与捕捉目标实例具体外观的视觉表示之间存在语义鸿沟,常造成掩码不完整和非目标区域误判。受感知锚定中符号-感知对应关系启发,本文提出原型引导文本校准(PTC)。在感知阶段,PTC根据初始匹配分数选取可靠视觉证据,构建类别特定的视觉原型;在锚定阶段,利用这些原型校准对应文本嵌入,校准强度根据视觉证据量自适应调整。由此,校准后的文本嵌入更准确地对齐实例级视觉表示,同时保留通用类别语义和开放词汇泛化能力。PTC无需额外训练或外部模型,可作为即插即用模块集成至现有方法。在八个基准上的实验表明,PTC显著提升六种代表性方法的性能,生成更完整、更精确的分割结果。验证了PTC是一种简单有效的视觉-文本对齐改进方法。

原文摘要 · Abstract (English)

Training-free open-vocabulary semantic segmentation (OVSS) partitions an image into semantically distinct regions based on arbitrary text descriptions, without learning any additional parameters. However, existing methods typically focus on improving visual representations while treating text embeddings that encode only generic category concepts as fixed classification references. The resulting semantic gap between these generic concepts and the visual representations that capture the specific appearances of target instances often causes incomplete masks and erroneous predictions in non-target regions. Inspired by the symbol-percept correspondence underlying perceptual anchoring, we propose Prototype-Guided Text Calibration (PTC) for training-free OVSS. In the Perceiving stage, PTC selects reliable visual evidence based on initial matching scores to construct category-specific visual prototypes. In the Anchoring stage, PTC uses these prototypes to calibrate their corresponding text embeddings, with the calibration strength adaptively adjusted based on the amount of visual evidence. Consequently, the calibrated text embeddings align more accurately with instance-specific visual representations while preserving generic category semantics and open-vocabulary generalization. Moreover, PTC requires neither additional training nor external models and can serve as a plug-and-play module for existing methods. Extensive experiments across eight benchmarks show that PTC significantly enhances the performance of six representative methods and yields more complete and accurate segmentation results. These results validate PTC as a simple and effective approach to improving visual-text alignment.

开放词汇语义分割文本校准无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。