arXiv:2512.11260cs.CV2025-12

对比Reformer与ViT在视觉任务中的表现,发现后者更高效。

Do We Need Reformer for Vision? An Experimental Comparison with Vision Transformers

  • 用局部敏感哈希近似全局自注意力,降低计算复杂度
  • 在小数据集上准确率略高,但大图上速度更慢
  • 适合研究注意力机制效率,不推荐实际部署

Transformer在计算机视觉中表现优异,但标准视觉变压器(ViT)因全局自注意力随标记数平方增长而计算开销大,限制了其在高分辨率输入和资源受限场景的应用。本文探索Reformer架构作为视觉骨干网络的可行性,结合基于块的标记化与局部敏感哈希(LSH)注意力,在序列长度n下将理论时间复杂度从$/mathcal{O}(n^2)$降至$/mathcal{O}(n \log n)$。我们在CIFAR-10上评估小规模数据集表现,在ImageNet-100上分析精度-效率权衡,并在高分辨率医学影像数据集上测试长序列下的性能。结果表明,虽然Reformer在CIFAR-10上准确率高于基线ViT,但在更大、更高分辨率设置下,ViT始终在实际效率和端到端计算时间上优于Reformer。这说明尽管LSH注意力具有理论优势,但显著的计算提升需依赖远超典型高分辨率图像生成的序列长度。

原文摘要 · Abstract (English)

Transformers have recently demonstrated strong performance in computer vision, with Vision Transformers (ViTs) leveraging self-attention to capture both low-level and high-level image features. However, standard ViTs remain computationally expensive, since global self-attention scales quadratically with the number of tokens, which limits their practicality for high-resolution inputs and resource-constrained settings. In this work, we investigate the Reformer architecture as an alternative vision backbone. By combining patch-based tokenization with locality-sensitive hashing (LSH) attention, our model approximates global self-attention while reducing its theoretical time complexity from $\mathcal{O}(n^2)$ to $\mathcal{O}(n \log n)$ in the sequence length $n$. We evaluate the proposed Reformer-based vision model on CIFAR-10 to assess its behavior on small-scale datasets, on ImageNet-100 to study its accuracy--efficiency trade-off in a more realistic setting, and on a high-resolution medical imaging dataset to evaluate the model under longer token sequences. While the Reformer achieves higher accuracy on CIFAR-10 compared to our ViT-style baseline, the ViT model consistently outperforms the Reformer in our experiments in terms of practical efficiency and end-to-end computation time across the larger and higher-resolution settings. These results suggest that, despite the theoretical advantages of LSH-based attention, meaningful computation gains require sequence lengths substantially longer than those produced by typical high-resolution images.

视觉Transformer注意力机制计算效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。