针对视觉Transformer的计算瓶颈，提出低功耗加速器设计。
Low Power Vision Transformer Accelerator with Hardware-Aware Pruning and Optimized Dataflow
- 通过动态标记剪枝与激活函数替换降低模型复杂度
- 运算量减少61.5%，FFN2权重减少59.3%，精度损失<2%
- 支持动态操作，能效达2.31 TOPS/W，适合边缘部署
当前Transformer加速器主要优化自注意力机制，但对短序列的视觉Transformer而言，前馈网络（FFN）才是主要计算瓶颈。本文提出一种低功耗视觉Transformer加速器，通过算法-硬件协同设计实现优化。采用硬件友好的动态标记剪枝减少模型复杂度，用ReLU替代GELU并结合动态FFN2剪枝，使运算量减少61.5%，FFN2权重减少59.3%，精度损失小于2%。硬件采用逐行数据流和输出导向访问策略，避免数据转置，支持动态操作且面积开销小。在台积电28nm CMOS工艺下实现，占用496.4K门，配备232KB SRAM缓冲区，峰值吞吐1024 GOPS（1GHz），能效2.31 TOPS/W，面积效率858.61 GOPS/mm²。
原文摘要 · Abstract (English)
Current transformer accelerators primarily focus on optimizing self-attention due to its quadratic complexity. However, this focus is less relevant for vision transformers with short token lengths, where the Feed-Forward Network (FFN) tends to be the dominant computational bottleneck. This paper presents a low power Vision Transformer accelerator, optimized through algorithm-hardware co-design. The model complexity is reduced using hardware-friendly dynamic token pruning without introducing complex mechanisms. Sparsity is further improved by replacing GELU with ReLU activations and employing dynamic FFN2 pruning, achieving a 61.5\% reduction in operations and a 59.3\% reduction in FFN2 weights, with an accuracy loss of less than 2\%. The hardware adopts a row-wise dataflow with output-oriented data access to eliminate data transposition, and supports dynamic operations with minimal area overhead. Implemented in TSMC's 28nm CMOS technology, our design occupies 496.4K gates and includes a 232KB SRAM buffer, achieving a peak throughput of 1024 GOPS at 1GHz, with an energy efficiency of 2.31 TOPS/W and an area efficiency of 858.61 GOPS/mm2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。