通过融合图像块上下文与增强类别令牌,提升弱监督语义分割精度
Context Patch Fusion With Class Token Enhancement for Weakly Supervised Semantic Segmentation
- 用双向长短期记忆模块捕捉图像块间空间依赖关系
- 引入可学习类别令牌动态优化类别特征表示,提升区分度
- 在PASCAL VOC和MS COCO上优于现有方法,适合弱监督场景
弱监督语义分割(WSSS)仅依赖图像级标签,因其低成本与可扩展性受到关注。现有方法主要增强类间差异并使用数据增强缓解语义模糊与虚假激活,但常忽略图像块间的复杂上下文依赖,导致局部表征不完整、分割精度受限。为此,本文提出上下文块融合与类别令牌增强(CPF-CTE)框架,利用块间上下文关系丰富特征表示。核心的上下文融合双向长短期记忆(CF-BiLSTM)模块捕获块间空间依赖并实现双向信息流动,增强特征学习与分割鲁棒性。同时引入可学习类别令牌,动态编码与精炼类别特定语义,提升判别能力。通过有效融合空间与语义线索,CPF-CTE生成更丰富准确的图像内容表征。在PASCAL VOC 2012和MS COCO 2014上的大量实验表明,该方法持续超越现有WSSS方法。
原文摘要 · Abstract (English)
Weakly Supervised Semantic Segmentation (WSSS), which relies only on image-level labels, has attracted significant attention for its cost-effectiveness and scalability. Existing methods mainly enhance inter-class distinctions and employ data augmentation to mitigate semantic ambiguity and reduce spurious activations. However, they often neglect the complex contextual dependencies among image patches, resulting in incomplete local representations and limited segmentation accuracy. To address these issues, we propose the Context Patch Fusion with Class Token Enhancement (CPF-CTE) framework, which exploits contextual relations among patches to enrich feature representations and improve segmentation. At its core, the Contextual-Fusion Bidirectional Long Short-Term Memory (CF-BiLSTM) module captures spatial dependencies between patches and enables bidirectional information flow, yielding a more comprehensive understanding of spatial correlations. This strengthens feature learning and segmentation robustness. Moreover, we introduce learnable class tokens that dynamically encode and refine class-specific semantics, enhancing discriminative capability. By effectively integrating spatial and semantic cues, CPF-CTE produces richer and more accurate representations of image content. Extensive experiments on PASCAL VOC 2012 and MS COCO 2014 validate that CPF-CTE consistently surpasses prior WSSS methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。