arXiv:2507.05838cs.CV2025-07

提出双原型机制与定向掩码,提升少样本分割的准确性和鲁棒性。

Inter- and Intra-image Refinement for Few Shot Segmentation

  • 用双原型捕捉类别一致特征与局部特异性特征,缓解类内差异。
  • 引入方向性丢弃机制,过滤不一致的跨注意力对,提升解码器性能。
  • 在9个基准上达到当前最优,适用于标准、部件及跨域少样本分割任务。

语义分割的深度神经网络依赖大规模标注数据,导致标注瓶颈,催生了少样本语义分割(FSS)——旨在仅用少量标注样本泛化到新类别。现有FSS方法多采用基于原型的方法,通过支持图像中掩码区域的特征生成查询先验图并指导预测。然而,其受图像间与图像内差异影响:1)单原型表示导致支持与查询图像间类内差距,使先验图分散且噪声大;2)视觉相似但语义不同的区域造成类间干扰,导致支持-查询特征匹配不一致,引发错误预测。为此,我们提出互-内图像精炼(IIR)模型。该模型包含基于图像间类别激活映射的方法,生成两个原型用于类别一致区域匹配,包括核心判别特征与局部特异性特征,从而生成更准确、鲁棒的先验图。针对图像内精炼,引入方向性丢弃机制,在交叉注意力中屏蔽不一致的支持-查询特征对,提升解码器表现。大量实验表明,IIR在9个基准上实现最先进性能,涵盖标准FSS、部件FSS与跨域FSS。代码已公开于 https://github.com/forypipi/IIR。

原文摘要 · Abstract (English)

Deep neural networks for semantic segmentation rely on large-scale annotated datasets, leading to an annotation bottleneck that motivates few shot semantic segmentation (FSS) which aims to generalize to novel classes with minimal labeled exemplars. Most existing FSS methods adopt a prototype-based paradigm, which generates query prior map by extracting masked-area features from support images and then makes predictions guided by the prior map. However, they suffer from two critical limitations induced by inter- and intra-image discrepancies: 1) The intra-class gap between support and query images, caused by single-prototype representation, results in scattered and noisy prior maps; 2) The inter-class interference from visually similar but semantically distinct regions leads to inconsistent support-query feature matching and erroneous predictions. To address these issues, we propose the Inter- and Intra-image Refinement (IIR) model. The model contains an inter-image class activation mapping based method that generates two prototypes for class-consistent region matching, including core discriminative features and local specific features, and yields an accurate and robust prior map. For intra-image refinement, a directional dropout mechanism is introduced to mask inconsistent support-query feature pairs in cross attention, thereby enhancing decoder performance. Extensive experiments demonstrate that IIR achieves state-of-the-art performance on 9 benchmarks, covering standard FSS, part FSS, and cross-domain FSS. Our source code is available at \href{https://github.com/forypipi/IIR}{https://github.com/forypipi/IIR}.

少样本分割图像精炼原型学习注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。