专为边缘设备优化的二值化Transformer加速器，能效提升311倍。
COBRA: Algorithm-Architecture Co-optimized Binary Transformer Accelerator for Edge Inference
- 设计专用1比特乘法单元，支持-1/0/+1矩阵运算，优于传统三值方法。
- 在边缘FPGA上实现3894.7 GOPS吞吐与448.7 GOPS/W能效，较GPU提升311倍。
- 适合低功耗边缘部署场景，尤其对能效敏感的实时推理任务。
基于Transformer的模型在自然语言处理和计算机视觉等领域表现卓越，但其庞大的模型规模及高计算、内存与通信需求限制了在边缘平台的本地安全推理部署。二值化Transformer通过降低模型复杂度和带宽需求，为边缘部署提供了紧凑解决方案，且保持可接受的精度。然而，现有二值化Transformer因缺乏针对硬件的优化，在当前硬件上运行效率低下。为此，本文提出COBRA，一种面向边缘计算的算法-架构协同优化的二值化Transformer加速器。COBRA采用真实的1比特二值乘法单元，支持-1、0、+1取值的矩阵运算，超越了三值方法。结合注意力模块的硬件友好优化，COBRA在边缘FPGA上实现了最高3,894.7 GOPS的吞吐量和448.7 GOPS/W的能效，相比GPU提升了311倍，相较当前最优二值化加速器吞吐提升3.5倍，仅带来可忽略的推理精度损失。
原文摘要 · Abstract (English)
Transformer-based models have demonstrated superior performance in various fields, including natural language processing and computer vision. However, their enormous model size and high demands in computation, memory, and communication limit their deployment to edge platforms for local, secure inference. Binary transformers offer a compact, low-complexity solution for edge deployment with reduced bandwidth needs and acceptable accuracy. However, existing binary transformers perform inefficiently on current hardware due to the lack of binary specific optimizations. To address this, we introduce COBRA, an algorithm-architecture co-optimized binary Transformer accelerator for edge computing. COBRA features a real 1-bit binary multiplication unit, enabling matrix operations with -1, 0, and +1 values, surpassing ternary methods. With further hardware-friendly optimizations in the attention block, COBRA achieves up to 3,894.7 GOPS throughput and 448.7 GOPS/Watt energy efficiency on edge FPGAs, delivering a 311x energy efficiency improvement over GPUs and a 3.5x throughput improvement over the state-of-the-art binary accelerator, with only negligible inference accuracy degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。