用图像级标注实现少样本语义分割，性能领先
AFANet: Adaptive Frequency-Aware Network for Weakly-Supervised Few-Shot Semantic Segmentation
- 提出跨粒度频域感知模块，分离高低频特征优化结构信息
- 引入在线学习的文本空间适配器，提升跨模态语义融合效果
- 在Pascal-5i和COCO-20i上达到最新最优结果，适合弱监督场景
少样本学习旨在利用少量样本识别新概念。然而，对于视觉密集型任务如少样本语义分割，像素级标注耗时且成本高。为此，本文采用更具挑战性的图像级标注，提出自适应频域感知网络（AFANet），用于弱监督少样本语义分割（WFSS）。具体地，提出跨粒度频域感知模块（CFM），将RGB图像分解为高频与低频分布，并通过重对齐优化语义结构信息。不同于多数现有方法离线使用多模态模型（如CLIP）的文本信息，本文进一步设计基于CLIP的时空适配模块（CSM），通过在线学习对文本信息进行空间域自适应变换，从而为CFM提供更丰富的跨模态语义信息。在Pascal-5i和COCO-20i数据集上的大量实验表明，AFANet实现了当前最优性能。代码已开源：https://github.com/jarch-ma/AFANet。
原文摘要 · Abstract (English)
Few-shot learning aims to recognize novel concepts by leveraging prior knowledge learned from a few samples. However, for visually intensive tasks such as few-shot semantic segmentation, pixel-level annotations are time-consuming and costly. Therefore, in this paper, we utilize the more challenging image-level annotations and propose an adaptive frequency-aware network (AFANet) for weakly-supervised few-shot semantic segmentation (WFSS). Specifically, we first propose a cross-granularity frequency-aware module (CFM) that decouples RGB images into high-frequency and low-frequency distributions and further optimizes semantic structural information by realigning them. Unlike most existing WFSS methods using the textual information from the multi-modal language-vision model, e.g., CLIP, in an offline learning manner, we further propose a CLIP-guided spatial-adapter module (CSM), which performs spatial domain adaptive transformation on textual information through online learning, thus providing enriched cross-modal semantic information for CFM. Extensive experiments on the Pascal-5\textsuperscript{i} and COCO-20\textsuperscript{i} datasets demonstrate that AFANet has achieved state-of-the-art performance. The code is available at https://github.com/jarch-ma/AFANet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。