用Transformer设计多尺度解码器，提升少样本语义分割精度与效率
MSDNet: Multi-Scale Decoder for Few-Shot Semantic Segmentation via Transformer-Guided Prototyping
- 基于Transformer构建空间注意力解码器，增强支持图与查询图关联
- 多尺度解码融合多分辨率特征，1-shot和5-shot下均超越主流方法
- 仅150万参数却兼顾性能与轻量，适合资源受限场景部署
少样本语义分割需在极少标注样本下完成图像分割。现有方法或忽略局部语义细节，或计算复杂度高。本文提出基于Transformer的框架，引入空间变换解码器与上下文掩码生成模块，增强支持图与查询图间的语义关系理解；设计多尺度解码器，分层融合不同分辨率特征以优化分割掩码；同时整合编码器中间层的全局特征以增强上下文感知，保持轻量结构降低计算开销。该方法在PASCAL-5^i与COCO-20^i数据集上，1-shot和5-shot设置下均取得竞争力表现，模型仅含150万参数，有效克服了现有方法的局限性。
原文摘要 · Abstract (English)
Few-shot Semantic Segmentation addresses the challenge of segmenting objects in query images with only a handful of annotated examples. However, many previous state-of-the-art methods either have to discard intricate local semantic features or suffer from high computational complexity. To address these challenges, we propose a new Few-shot Semantic Segmentation framework based on the Transformer architecture. Our approach introduces the spatial transformer decoder and the contextual mask generation module to improve the relational understanding between support and query images. Moreover, we introduce a multi scale decoder to refine the segmentation mask by incorporating features from different resolutions in a hierarchical manner. Additionally, our approach integrates global features from intermediate encoder stages to improve contextual understanding, while maintaining a lightweight structure to reduce complexity. This balance between performance and efficiency enables our method to achieve competitive results on benchmark datasets such as PASCAL-5^i and COCO-20^i in both 1-shot and 5-shot settings. Notably, our model with only 1.5 million parameters demonstrates competitive performance while overcoming limitations of existing methodologies. https://github.com/amirrezafateh/MSDNet
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。