通过全局特征增强提升Transformer模型在噪声下的分割鲁棒性。
Global Average Feature Augmentation for Robust Semantic Segmentation with Transformers
- 引入通道级全局扰动增强，训练时开销极小。
- 在Cityscapes上噪声下分割精度提升27.7%，达84.3%保留率。
- 适用于追求鲁棒性的视觉分割场景，尤其适合部署于复杂环境。
现有视觉变换器（如SegFormer）在应对设备采集带来的模糊、噪声等分布外干扰时表现出良好鲁棒性。本文提出通道级特征增强（CWFA），一种简单高效的特征增强方法，用于进一步提升视觉变换器在语义分割任务中的鲁棒性。CWFA在每个编码器中施加全局估计的扰动，训练时计算开销极低。在Cityscapes和ADE20K数据集上，对SegFormer、Swin Transformer和Twins三种先进视觉变换器架构进行大量实验验证，结果显示，采用CWFA增强的模型在不损害原始数据性能的前提下显著提升鲁棒性。例如，在Cityscapes上，经过CWFA增强的SegFormer-B1模型在脉冲噪声下相比未增强版本，平均交并比（mIoU）提升高达27.7%；此外，增强后的SegFormer-B5达到84.3%的保留率，较最新FAN+STL方法提升0.7%，刷新当前最佳表现。
原文摘要 · Abstract (English)
Robustness to out-of-distribution data is crucial for deploying modern neural networks. Recently, Vision Transformers, such as SegFormer for semantic segmentation, have shown impressive robustness to visual corruptions like blur or noise affecting the acquisition device. In this paper, we propose Channel Wise Feature Augmentation (CWFA), a simple yet efficient feature augmentation technique to improve the robustness of Vision Transformers for semantic segmentation. CWFA applies a globally estimated perturbation per encoder with minimal compute overhead during training. Extensive evaluations on Cityscapes and ADE20K, with three state-of-the-art Vision Transformer architectures : SegFormer, Swin Transformer, and Twins demonstrate that CWFA-enhanced models significantly improve robustness without affecting clean data performance. For instance, on Cityscapes, a CWFA-augmented SegFormer-B1 model yields up to 27.7% mIoU robustness gain on impulse noise compared to the non-augmented SegFormer-B1. Furthermore, CWFA-augmented SegFormer-B5 achieves a new state-of-the-art 84.3% retention rate, a 0.7% improvement over the recently published FAN+STL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。