arXiv:2505.05023cs.CV2025-05中稿 · BMVC 2025被引 4

提出解耦匹配策略,提升零样本分割中未见类的识别准确率。

Split Matching for Inductive Zero-shot Semantic Segmentation

  • 将匈牙利匹配拆分为已见类和潜在未见类两部分独立优化
  • 在两个基准上达到当前最优性能,显著减少未见类误判为背景
  • 适合研究零样本视觉分割与提示学习的学者参考

零样本语义分割(ZSS)旨在分割训练中未标注的类别。尽管微调视觉-语言模型已取得良好效果,但这些模型常因缺乏未见类监督而过拟合于已见类别。作为全监督方法的替代,基于查询的分割在ZSS中展现出巨大潜力,因其可在无需显式标签的情况下实现目标定位。然而,传统匈牙利匹配作为查询框架的核心组件,需要完整监督,在ZSS设定下常将未见类别误判为背景。为此,我们提出分拆匹配(Split Matching, SM),一种新颖的分配策略,将匈牙利匹配解耦为两部分:一部分用于标注区域中的已见类别,另一部分用于未标注区域中的潜在未见类(称为未见候选)。具体而言,我们将查询分为已见组与候选组,使其根据可用监督独立优化。为发现未见候选,我们对CLIP稠密特征进行聚类生成伪掩码,并使用CLS token提取区域级嵌入。随后基于类别级相似性和掩码级一致性,分别对两组进行匹配。此外,我们引入多尺度特征增强(MFE)模块,通过残差多尺度聚合细化解码器特征,提升模型在多分辨率下捕捉空间细节的能力。SM是首个在归纳式零样本分割设定下引入解耦匈牙利匹配的方法,在两个标准基准上均达到当前最优性能。

原文摘要 · Abstract (English)

Zero-shot Semantic Segmentation (ZSS) aims to segment categories that are not annotated during training. While fine-tuning vision-language models has achieved promising results, these models often overfit to seen categories due to the lack of supervision for unseen classes. As an alternative to fully supervised approaches, query-based segmentation has shown great latent in ZSS, as it enables object localization without relying on explicit labels. However, conventional Hungarian matching, a core component in query-based frameworks, needs full supervision and often misclassifies unseen categories as background in the setting of ZSS. To address this issue, we propose Split Matching (SM), a novel assignment strategy that decouples Hungarian matching into two components: one for seen classes in annotated regions and another for latent classes in unannotated regions (referred to as unseen candidates). Specifically, we partition the queries into seen and candidate groups, enabling each to be optimized independently according to its available supervision. To discover unseen candidates, we cluster CLIP dense features to generate pseudo masks and extract region-level embeddings using CLS tokens. Matching is then conducted separately for the two groups based on both class-level similarity and mask-level consistency. Additionally, we introduce a Multi-scale Feature Enhancement (MFE) module that refines decoder features through residual multi-scale aggregation, improving the model's ability to capture spatial details across resolutions. SM is the first to introduce decoupled Hungarian matching under the inductive ZSS setting, and achieves state-of-the-art performance on two standard benchmarks.

零样本分割视觉语言模型查询匹配图像分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。