用双视角网络提升弱监督少样本分割性能，参数更少效果更好
Through the Looking Glass: A Dual Perspective on Weakly-Supervised Few-Shot Segmentation
- 将支持与查询图像视为双视角，设计异质视觉聚合模块增强互补性
- 参数仅需现有模型一半，在Pascal-5i上提升13.2%，COCO-20i上提升9.7%
- 首个弱监督模型超越同架构全监督模型，适合资源受限场景
元学习通常均匀采样同类别的支持-查询对，并通过相同网络结构提取归纳偏置，但会导致语义过度同质化。为此，我们提出一种同源异质网络，将支持-查询对视为双视角，引入异质视觉聚合（HA）模块增强互补性，同时保持语义共性；为减少语义噪声并强化异质语义独特性，设计了异质迁移（HT）模块；最后提出异质CLIP（HC）文本信息，提升多模态模型泛化能力。在弱监督少样本语义分割（WFSS）任务中，TLG仅需现有先进模型1/24的参数量，便在Pascal-5i上实现13.2%的提升，在COCO-20i上提升9.7%。据我们所知，TLG是首个在相同骨干网络下，弱监督（图像级标注）性能超过全监督（像素级标注）的模型。代码已公开于https://github.com/jarch-ma/TLG。
原文摘要 · Abstract (English)
Meta-learning aims to uniformly sample homogeneous support-query pairs, characterized by the same categories and similar attributes, and extract useful inductive biases through identical network architectures. However, this identical network design results in over-semantic homogenization. To address this, we propose a novel homologous but heterogeneous network. By treating support-query pairs as dual perspectives, we introduce heterogeneous visual aggregation (HA) modules to enhance complementarity while preserving semantic commonality. To further reduce semantic noise and amplify the uniqueness of heterogeneous semantics, we design a heterogeneous transfer (HT) module. Finally, we propose heterogeneous CLIP (HC) textual information to enhance the generalization capability of multimodal models. In the weakly-supervised few-shot semantic segmentation (WFSS) task, with only 1/24 of the parameters of existing state-of-the-art models, TLG achieves a 13.2\% improvement on Pascal-5\textsuperscript{i} and a 9.7\% improvement on COCO-20\textsuperscript{i}. To the best of our knowledge, TLG is also the first weakly supervised (image-level) model that outperforms fully supervised (pixel-level) models under the same backbone architectures. The code is available at https://github.com/jarch-ma/TLG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。