提出无混叠视觉Transformer，提升对图像平移的鲁棒性。
Alias-Free ViT: Fractional Shift Invariance via Linear Attention
- 用无混叠下采样和非线性层替代传统模块
- 线性交叉协方差注意力支持整数与分数平移等变性
- 在分类精度相当下，对抗平移攻击更鲁棒
Transformer在视觉任务中已成为卷积网络的有力竞争者，但缺乏卷积网络的结构先验，可能限制其性能。具体而言，视觉Transformer（ViTs）不具备平移不变性，对图像微小平移比标准卷积网络更敏感。尽管已有研究指出卷积网络也因下采样和非线性层的混叠问题而并非完全平移不变，因此提出了抗混叠方法以保证其平移鲁棒性。受此启发，本文提出无混叠视觉Transformer（Alias-Free ViT），包含两个核心组件：一是采用无混叠下采样与非线性变换；二是使用线性交叉协方差注意力，该注意力机制对整数和分数平移均具有等变性，从而实现全局平移不变表示。模型在图像分类任务上保持竞争力，并在相似规模下对对抗性平移攻击表现出更强鲁棒性。
原文摘要 · Abstract (English)
Transformers have emerged as a competitive alternative to convnets in vision tasks, yet they lack the architectural inductive bias of convnets, which may hinder their potential performance. Specifically, Vision Transformers (ViTs) are not translation-invariant and are more sensitive to minor image translations than standard convnets. Previous studies have shown, however, that convnets are also not perfectly shift-invariant, due to aliasing in downsampling and nonlinear layers. Consequently, anti-aliasing approaches have been proposed to certify convnets' translation robustness. Building on this line of work, we propose an Alias-Free ViT, which combines two main components. First, it uses alias-free downsampling and nonlinearities. Second, it uses linear cross-covariance attention that is shift-equivariant to both integer and fractional translations, enabling a shift-invariant global representation. Our model maintains competitive performance in image classification and outperforms similar-sized models in terms of robustness to adversarial translations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。