arXiv:2502.15885cs.CV2025-02被引 5

通过优化嵌入信息提升注意力激活图质量,改善弱监督语义分割效果。

DOEI: Dual Optimization of Embedding Information for Attention-Enhanced Class Activation Maps

  • 引入双优化机制,用语义感知注意力重构建模特征表示。
  • 在PASCAL VOC和MS COCO上分别提升3.6%、1.2% mIoU。
  • 适合作为插件模块集成到视觉变换器模型中使用。

弱监督语义分割(WSSS)通常依赖有限的语义标注生成初始类别激活图(CAM)。然而,由于高维空间中类别激活响应与语义信息耦合不足,导致CAM易受物体共现或激活不足影响,识别精度受限。为此,本文提出DOEI(Dual Optimization of Embedding Information),通过语义感知注意力权重矩阵重构嵌入表示,优化嵌入信息表达能力。具体地,在类别-图像块交互中增强高置信度令牌、抑制低置信度令牌,使激活响应与语义信息对齐,强化目标特征的传播与解耦,使高阶语义空间中的嵌入更准确反映目标特征。此外,设计融合RGB值、嵌入引导特征与自注意力权重的混合特征对齐模块,提高候选令牌可靠性。大量实验表明,DOEI是有效的即插即用模块,显著提升基于视觉变换器的先进WSSS模型在主流基准上的CAM质量与分割性能,在PASCAL VOC上分别提升3.6%、1.5%、1.2% mIoU,MS COCO上提升1.2%、1.6% mIoU。代码将开源至https://github.com/AIGeeksGroup/DOEI。

原文摘要 · Abstract (English)

Weakly supervised semantic segmentation (WSSS) typically utilizes limited semantic annotations to obtain initial Class Activation Maps (CAMs). However, due to the inadequate coupling between class activation responses and semantic information in high-dimensional space, the CAM is prone to object co-occurrence or under-activation, resulting in inferior recognition accuracy. To tackle this issue, we propose DOEI, Dual Optimization of Embedding Information, a novel approach that reconstructs embedding representations through semantic-aware attention weight matrices to optimize the expression capability of embedding information. Specifically, DOEI amplifies tokens with high confidence and suppresses those with low confidence during the class-to-patch interaction. This alignment of activation responses with semantic information strengthens the propagation and decoupling of target features, enabling the generated embeddings to more accurately represent target features in high-level semantic space. In addition, we propose a hybrid-feature alignment module in DOEI that combines RGB values, embedding-guided features, and self-attention weights to increase the reliability of candidate tokens. Comprehensive experiments show that DOEI is an effective plug-and-play module that empowers state-of-the-art visual transformer-based WSSS models to significantly improve the quality of CAMs and segmentation performance on popular benchmarks, including PASCAL VOC (+3.6%, +1.5%, +1.2% mIoU) and MS COCO (+1.2%, +1.6% mIoU). Code will be available at https://github.com/AIGeeksGroup/DOEI.

弱监督分割注意力机制特征对齐Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。