针对视觉Transformer设计了词元级数据增强，提升半监督语义分割性能。
Exploring Token-Level Augmentation in Vision Transformer for Semi-Supervised Semantic Segmentation
- 在词元层面混合图像，适配Transformer的全局注意力机制。
- 在有限标注下显著提升分割精度，尤其在Pascal VOC和Cityscapes上超越当前最佳。
- 适合研究视觉Transformer与半监督学习结合的学者参考。
半监督语义分割近年来取得显著进展，但现有方法多基于卷积神经网络，直接应用于视觉变换器时因概念差异存在局限。为此，我们提出TokenMix，一种专为视觉变换器设计的半监督语义分割数据增强技术。TokenMix通过在词元层面混合图像，契合全局注意力机制，增强图像块间的上下文信息学习能力。进一步引入图像增强与特征增强，提升多样性。同时，提出双分支框架，每个分支对输入图像分别应用图像与特征增强，强化一致性正则化。在Pascal VOC 2012、Cityscapes和COCO等多个基准数据集上进行大量实验，结果表明该方法在有限精细标注条件下优于现有最先进算法，表现出显著精度提升。
原文摘要 · Abstract (English)
Semi-supervised semantic segmentation has witnessed remarkable advancements in recent years. However, existing algorithms are based on convolutional neural networks and directly applying them to Vision Transformers poses certain limitations due to conceptual disparities. To this end, we propose TokenMix, a data augmentation technique specifically designed for semi-supervised semantic segmentation with Vision Transformers. TokenMix aligns well with the global attention mechanism by mixing images at the token level, enhancing learning capability for contextual information among image patches. We further incorporate image augmentation and feature augmentation to promote the diversity of augmentation. Moreover, to enhance consistency regularization, we propose a dual-branch framework where each branch applies image and feature augmentation to the input image. We conduct extensive experiments across multiple benchmark datasets, including Pascal VOC 2012, Cityscapes, and COCO. Results suggest that the proposed method outperforms state-of-the-art algorithms with notably observed accuracy improvement, especially under limited fine annotations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。