FPGA加速Transformer矩阵乘法，效率比CPU快7倍、比普通计算快200倍。
Design and Implementation of an FPGA-Based Hardware Accelerator for Transformer
- 采用分块存储与两级分块策略，最大化数据复用率。
- 在100MHz下实现3.1 GFLOPs吞吐，64×768与768×3072矩阵乘达200倍加速。
- 适合边缘设备上高效部署大模型推理，尤其对资源受限场景友好。
基于Transformer的大语言模型严重依赖密集的矩阵乘法，其中多头自注意力（MHA）模块中的Q、K、V线性投影构成关键性能瓶颈。本文设计并实现了一款面向资源受限Xilinx KV260 FPGA的优化分块矩阵乘法加速器，通过持久化片上存储、稳健的两级分块策略及类似流水线的展开计算单元，显著提升速度与能效。集成DistilBERT的Q、K、V投影后，相比ARM CPU（PyTorch）实现7倍加速，相比朴素NumPy达到200倍提升，在100MHz时钟下对(64,768)×(768,3072)矩阵乘法最高达3.1 GFLOPs吞吐。结果充分证明了FPGA加速在关键Transformer操作上的变革潜力，为边缘设备上可扩展、低功耗的深度学习推理开辟路径。
原文摘要 · Abstract (English)
Transformer-based large language models (LLMs) rely heavily on intensive matrix multiplications for attention and feed-forward layers, with the Q, K, and V linear projections in the Multi-Head Self-Attention (MHA) module constituting a decisive performance bottleneck. In this work, we introduce a highly optimized tiled matrix multiplication accelerator on a resource-constrained Xilinx KV260 FPGA that not only addresses this challenge but sets a new standard for efficiency and performance. Our design exploits persistent on-chip storage, a robust two-level tiling strategy for maximal data reuse, and a systolic-like unrolled compute engine that together deliver unparalleled speed and energy efficiency. Integrated with DistilBERT for Q, K, and V projections, our accelerator achieves an unequivocal 7x speedup over ARM CPU implementations (PyTorch) and an extraordinary 200x improvement over naive NumPy, reaching a throughput of up to 3.1~GFLOPs for matrix multiplications on (64,768) x (768,3072) matrices while operating at a conservative 100 MHz. These results decisively demonstrate the transformative potential of FPGA-based acceleration for critical Transformer operations, paving the way for scalable and energy-efficient deep learning inference on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。