解决视觉Transformer特征图噪声，提升分割与深度估计效果
Enhancing Transformer-Based Vision Models: Addressing Feature Map Anomalies Through Novel Optimization Strategies
- 通过空间扰动增强令牌多样性，缓解特征异常
- 在ImageNet、Ade20k等数据集上性能稳定提升
- 方法轻量且不依赖架构，适合各类视觉任务
视觉Transformer（ViTs）在众多计算机视觉任务中表现优异，但其特征图中存在的结构化噪声干扰了分割和深度估计等下游应用。本文提出两种新颖且轻量的优化策略：结构化令牌增强（STA）与自适应噪声过滤（ANF）。STA通过在令牌化阶段引入空间扰动，提升令牌多样性；ANF则在Transformer层间引入可学习的内联去噪机制。这两种方法均与架构无关，在ImageNet、Ade20k和NYUv2等标准基准上进行了评估。实验结果表明，该方法在视觉质量与任务性能上均有持续改进，验证了其实际有效性。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) have demonstrated superior performance across a wide range of computer vision tasks. However, structured noise artifacts in their feature maps hinder downstream applications such as segmentation and depth estimation. We propose two novel and lightweight optimisation techniques- Structured Token Augmentation (STA) and Adaptive Noise Filtering (ANF)- to improve interpretability and mitigate these artefacts. STA enhances token diversity through spatial perturbations during tokenisation, while ANF applies learnable inline denoising between transformer layers. These methods are architecture-agnostic and evaluated across standard benchmarks, including ImageNet, Ade20k, and NYUv2. Experimental results show consistent improvements in visual quality and task performance, highlighting the practical effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。