FPGA加速器FastMamba实现Mamba2高效部署,量化精度高、能效比优。
FastMamba: A High-Speed and Efficient Mamba Accelerator on FPGA with Accurate Quantization
- 通过哈达玛变换消除线性层异常值,实现8位精准量化
- 在VC709 FPGA上,130M模型推理速度比CPU快68.8倍,比GPU快8.9倍
- 适合边缘设备部署,2.7B模型能效比达显卡6倍,适合低功耗场景
状态空间模型(SSMs)如Mamba2已取得显著性能,但其在资源受限的边缘设备上部署面临挑战:线性层存在严重异常值影响量化,张量操作多样且不规则,SSM模块中的非线性函数硬件友好性差。为此,本文提出FastMamba,一种基于FPGA的专用加速器,采用软硬件协同设计提升Mamba2部署效率。具体而言,通过哈达玛变换成功实现线性层的8位量化;针对SSM和卷积层提出硬件友好的细粒度2的幂次量化框架,并采用一阶线性近似优化非线性函数。基于精确量化,设计集成并行向量处理单元、流水线执行数据流与高效非线性近似单元的加速器,提升计算效率并降低硬件复杂度。在Xilinx VC709 FPGA上评估:对于Mamba2-130M的输入预填充任务,相比Intel Xeon 4210R CPU提速68.80倍,相比NVIDIA RTX 3090 GPU提速8.90倍;在Mamba2-2.7B的输出解码实验中,能效比较RTX 3090 GPU高出6倍。
原文摘要 · Abstract (English)
State Space Models (SSMs), like recent Mamba2, have achieved remarkable performance and received extensive attention. However, deploying Mamba2 on resource-constrained edge devices encounters many problems: severe outliers within the linear layer challenging the quantization, diverse and irregular element-wise tensor operations, and hardware-unfriendly nonlinear functions in the SSM block. To address these issues, this paper presents FastMamba, a dedicated accelerator on FPGA with hardware-algorithm co-design to promote the deployment efficiency of Mamba2. Specifically, we successfully achieve 8-bit quantization for linear layers through Hadamard transformation to eliminate outliers. Moreover, a hardware-friendly and fine-grained power-of-two quantization framework is presented for the SSM block and convolution layer, and a first-order linear approximation is developed to optimize the nonlinear functions. Based on the accurate algorithm quantization, we propose an accelerator that integrates parallel vector processing units, pipelined execution dataflow, and an efficient SSM Nonlinear Approximation Unit, which enhances computational efficiency and reduces hardware complexity. Finally, we evaluate FastMamba on Xilinx VC709 FPGA. For the input prefill task on Mamba2-130M, FastMamba achieves 68.80\times and 8.90\times speedup over Intel Xeon 4210R CPU and NVIDIA RTX 3090 GPU, respectively. In the output decode experiment with Mamba2-2.7B, FastMamba attains 6\times higher energy efficiency than RTX 3090 GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。