arXiv:2606.08001cs.CV2026-06中稿 · 11th International…

通过建模类别间语义关系,提升开放词汇语义分割的识别精度。

Learning a Semantic Calibration Network for Open-Vocabulary Semantic Segmentation

论文配图:Learning a Semantic Calibration Network for Open-Vocabulary Semantic Segmentation
图 1 · 摘自论文原文
  • 引入语义校准网络,显式建模类别间依赖关系。
  • 在多个基准上超越现有方法,显著提升分割准确率。
  • 适合关注开放词汇理解与细粒度图像分割的研究者。

语义图像分割为每个像素分配预定义类别标签,近年来取得显著进展。开放词汇分割(OVS)将任务从固定类别集扩展至开放集,支持基于任意文本输入(如类别名或描述)识别并分割新概念。本文提出一种新型语义校准网络(SCN)用于开放词汇语义分割。不同于以往聚焦特征聚合或简单微调预训练模型的方法,SCN通过显式建模类别间的语义关联,优化掩码分类过程,增强模型判别力的同时有效保留CLIP模型的泛化能力。具体而言,SCN包含两个核心模块:类别消歧(CD)与逻辑融合(LF)。首先,利用交叉注意力机制将文本嵌入转化为视觉感知的伪文本嵌入,以生成增强的相似性分数,补充原始掩码-文本相似性分数;随后,类别消歧模块通过残差结构捕捉隐式的类间依赖关系,有效解决语义模糊问题;最后,逻辑融合模块动态整合多源语义证据,确保模型达成稳健的语义共识,同时保持CLIP固有的泛化能力。在主流基准上的全面实验表明,所提方法相比现有最优算法实现显著性能提升。

原文摘要 · Abstract (English)

Semantic image segmentation assigns a predefined category label to each pixel, has achieved significant progress lately. Open-Vocabulary Segmentation (OVS) extends the segmentation task from a fixed set to an open set, enabling the identification and segmentation of novel concepts based on arbitrary text inputs, such as category names or descriptions. In this paper, we propose a novel Semantic Calibration Network (SCN) for open-vocabulary semantic segmentation. Different from prior approaches that focus on feature aggregation or simple fine-tuning of pre-trained models, SCN refines the mask classification process by explicitly modeling the semantic correlations between classes, aiming to enhance the model's discriminative power while effectively preserving the generalization abilities of the pre-trained CLIP model. Specifically, SCN comprises two core components: Class Disambiguation (CD) and Logits Fusion (LF). First, a cross-attention mechanism is utilized to transform the text embeddings into visually aware pseudo-text embeddings, in order to derive an enhanced similarity score that complements the original mask-text similarity score. Subsequently, the Class Disambiguation module captures implicit inter-class dependencies through a residual architecture to effectively resolve semantic ambiguities. Finally, the Logits Fusion module dynamically integrates multifaceted semantic evidence to ensure that the model achieves a robust semantic consensus while maintaining CLIP's inherent generalization capability. Comprehensive experimental results on mainstream benchmarks demonstrate that the proposed method achieves significant performance improvements compared to state-of-the-art algorithms.

语义分割开放词汇CLIP视觉语言对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。