arXiv:2605.10661cs.CVcs.AI2026-05被引 1

用单块循环结构实现视觉Transformer,参数少十倍仍保持高精度。

bViT: Investigating Single-Block Recurrence in Vision Transformers for Image Recognition

论文配图:bViT: Investigating Single-Block Recurrence in Vision Transformers for Image Recognition
图 1 · 摘自论文原文
  • 单个Transformer块重复使用,替代多层独立参数设计。
  • 12步bViT-B在ImageNet-1K上达到与标准ViT-B相当的准确率。
  • 宽表示空间可实现隐式深度复用,适合高效微调和迁移任务。

视觉Transformer(ViTs)通过堆叠独立参数化的块构建,但其深度中有多大比例需要层特定变换,有多大比例可通过循环计算实现仍不明确。本文提出bViT,一种仅使用一个重复块的循环式ViT,保留深层ViT的迭代结构,同时消除层间参数差异,为研究循环机制提供可控环境。在ImageNet-1K上,12步的bViT-B在相同训练策略与计算预算下,性能接近标准ViT-B,参数量却减少一个数量级。观察发现,循环性能随表示宽度提升,更宽的bViT能恢复更多标准ViT性能。我们将其解释为隐式深度复用:共享块通过隐藏状态演化表达多步依赖计算。bViT在下游任务中表现良好,支持参数高效的微调。激活、注意力与步级剪枝的机制分析表明,共享块在不同循环步骤中表现出有效行为变化,而非简单重复。结果表明,只要表示空间足够宽,大量ViT深度可通过循环复用实现。

原文摘要 · Abstract (English)

Vision Transformers (ViTs) are built by stacking independently parameterized blocks, but it remains unclear how much of this depth requires layer specific transformations and how much can be realized through recurrent computation. We study this question with bViT, a single-block recurrent ViT in which one transformer block is applied repeatedly to process an image. This architecture preserves the iterative structure of a deep ViT while removing layer specific block parameterization, providing a controlled setting for studying recurrence in vision. On ImageNet-1K, a 12-step bViT-B achieves accuracy comparable to standard ViT-B under the same training recipe and computational budget, while using an order of magnitude fewer parameters. We observe that recurrent performance improves with representation width, with wider bViTs recovering much more of the performance of standard ViTs than narrow variants. We interpret this behavior as implicit depth multiplexing, where a shared block expresses multiple step-dependent computations through the evolving hidden state. Beyond ImageNet classification, bViT transfers competitively to downstream tasks and enables parameter-efficient fine-tuning. Mechanistic analyses of activations, attention and step-specific pruning show that the shared block changes its effective behavior across recurrent steps rather than simply repeating the same computation. Our results suggest that a large fraction of ViT depth can be implemented through recurrent reuse, provided that the representation space is sufficiently wide.

视觉Transformer循环结构参数效率深度复用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。