ViT-5通过优化组件提升视觉变压器性能，适配2020年代主流模型架构。
ViT-5: Vision Transformers for The Mid-2020s
- 逐项改进归一化、激活函数、位置编码等模块，形成新一代ViT结构。
- ImageNet-1k上达84.2%准确率，生成任务FID降至1.84，优于基线模型。
- 适合需要高效可靠视觉主干的科研与工业应用，可直接替换原有ViT。
本文系统研究了利用过去五年架构进展现代化视觉变压器骨干网络的方法。在保持经典注意力-前馈结构的基础上，对归一化、激活函数、位置编码、门控机制和可学习标记等组件进行逐项优化，形成新一代视觉变压器，称为ViT-5。大量实验表明，ViT-5在理解与生成任务中均持续优于当前最先进的纯视觉变压器。在ImageNet-1k分类任务中，ViT-5-Base在相近计算量下达到84.2%的顶级准确率，超过DeiT-III-Base的83.8%。当接入SiT扩散框架时，其生成性能也更优，FID降至1.84（原ViT为2.06）。除关键指标外，ViT-5展现出更强表征学习能力与合理的空间推理行为，且跨任务迁移稳定。其设计符合当代基础模型实践，为2020年代中期视觉骨干网络提供简单高效的即插即用升级方案。
原文摘要 · Abstract (English)
This work presents a systematic investigation into modernizing Vision Transformer backbones by leveraging architectural advancements from the past five years. While preserving the canonical Attention-FFN structure, we conduct a component-wise refinement involving normalization, activation functions, positional encoding, gating mechanisms, and learnable tokens. These updates form a new generation of Vision Transformers, which we call ViT-5. Extensive experiments demonstrate that ViT-5 consistently outperforms state-of-the-art plain Vision Transformers across both understanding and generation benchmarks. On ImageNet-1k classification, ViT-5-Base reaches 84.2\% top-1 accuracy under comparable compute, exceeding DeiT-III-Base at 83.8\%. ViT-5 also serves as a stronger backbone for generative modeling: when plugged into an SiT diffusion framework, it achieves 1.84 FID versus 2.06 with a vanilla ViT backbone. Beyond headline metrics, ViT-5 exhibits improved representation learning and favorable spatial reasoning behavior, and transfers reliably across tasks. With a design aligned with contemporary foundation-model practices, ViT-5 offers a simple drop-in upgrade over vanilla ViT for mid-2020s vision backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。