提出新型动态稀疏加速器，显著提升Transformer能效
LAPA: Log-Domain Prediction-Driven Dynamic Sparsity Accelerator for Transformer Model
- 在对数域预测注意力，减少乘法运算开销
- 多轮移位累加机制降低累积计算成本，提升精度
- 适合需要高能效的Transformer硬件部署场景
基于注意力的Transformer已革新自然语言处理，并在计算机视觉任务中表现出色。然而，随着输入序列变化，Transformer模型各阶段的计算瓶颈呈现动态特性，亟需跨阶段稀疏加速策略。现有大多数稀疏Transformer方法为单阶段设计，其稀疏性预测机制在多阶段应用时带来显著功耗开销。为此，本文提出一种对数域注意力预测-架构协同设计方法（LAPA）。首先，设计非对称首位1计算（ALOC）方案以消除昂贵的乘法操作；其次，提出混合精度多轮移位累加（MRSA）机制以缓解累积开销；同时设计数据-特征依赖滤波（DDF）策略与MRSA协同工作。最后，构建专用加速器将理论优势转化为实际硬件性能提升。实验表明，LAPA在能效上分别比当前最先进方法Spatten、Sanger和FACT高出3.52倍、3.24倍和2.79倍。
原文摘要 · Abstract (English)
Attention-based Transformers have revolutionized natural language processing (NLP) and shown strong performance in computer vision (CV) tasks. However, as the input sequence varies, the computational bottlenecks in Transformer models exhibit dynamic behavior across stages, which calls for a cross-stage sparse acceleration strategy. Unfortunately, most existing sparse Transformer approaches are single-stage based, and their sparsity prediction mechanisms lead to significant power overhead when applied across multiple stages. To this end, this paper proposes a log-domain attention prediction algorithm-architecture co-design, named LAPA. First, an asymmetric leading one computing (ALOC) scheme is designed to eliminate expensive multiplications. Next, a mixed-precision multi-round shifting accumulation (MRSA) mechanism is further proposed to mitigate the accumulation overhead. A data-feature dependent filter (DDF) strategy is designed to work in concert with the MRSA process. Finally, an elaborate accelerator is designed to translate the theoretical enhancement into practical hardware improvement. Experimental results show that LAPA achieves 3.52x, 3.24x and 2.79x higher energy efficiency than the state-of-the-art (SOTA) works Spatten, Sanger and FACT, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。