LoGoSeg通过融合局部与全局特征,实现更精准的开放词汇语义分割。
LoGoSeg: Integrating Local and Global Features for Open-Vocabulary Semantic Segmentation

- 引入物体存在先验与区域感知对齐,提升图文空间匹配精度。
- 在6个基准上达到领先性能,尤其在模糊场景中减少误分割。
- 单阶段设计无需额外掩码或模型,适合实时应用。
开放词汇语义分割(OVSS)通过任意文本描述扩展了传统封闭集分割,支持对已见和未见类别进行像素级标注。现有方法依赖如CLIP等视觉-语言模型(VLM),但其图像级预训练常导致空间对齐不精确,在模糊或复杂场景中产生错误分割。同时,多数方法缺乏强物体先验与区域约束,易引发物体幻觉或漏检。为此,本文提出LoGoSeg,一种高效单阶段框架,包含三项创新:(i) 物体存在先验,通过全局图文相似性动态加权相关类别,有效抑制幻觉;(ii) 区域感知对齐模块,建立精细的区域级图文对应关系;(iii) 双流融合机制,最优结合局部结构信息与全局语义上下文。相比先前工作,LoGoSeg无需外部掩码提议、额外主干网络或额外数据集,保障高效性。在六个基准(A-847, PC-459, A-150, PC-59, PAS-20, PAS-20b)上的大量实验表明,其在开放词汇设置下表现优异且泛化能力强。
原文摘要 · Abstract (English)
Open-vocabulary semantic segmentation (OVSS) extends traditional closed-set segmentation by enabling pixel-wise annotation for both seen and unseen categories using arbitrary textual descriptions. While existing methods leverage vision-language models (VLMs) like CLIP, their reliance on image-level pretraining often results in imprecise spatial alignment, leading to mismatched segmentations in ambiguous or cluttered scenes. However, most existing approaches lack strong object priors and region-level constraints, which can lead to object hallucination or missed detections, further degrading performance. To address these challenges, we propose LoGoSeg, an efficient single-stage framework that integrates three key innovations: (i) an object existence prior that dynamically weights relevant categories through global image-text similarity, effectively reducing hallucinations; (ii) a region-aware alignment module that establishes precise region-level visual-textual correspondences; and (iii) a dual-stream fusion mechanism that optimally combines local structural information with global semantic context. Unlike prior works, LoGoSeg eliminates the need for external mask proposals, additional backbones, or extra datasets, ensuring efficiency. Extensive experiments on six benchmarks (A-847, PC-459, A-150, PC-59, PAS-20, and PAS-20b) demonstrate its competitive performance and strong generalization in open-vocabulary settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。