arXiv:2409.11440cs.ARcs.AI2024-09中稿 · ICCAD 2024被引 39

MARCA通过可重构架构,大幅提升Mamba模型的推理速度与能效。

MARCA: Mamba Accelerator with ReConfigurable Architecture

  • 采用可重构计算阵列,动态切换线性与逐元素运算路径。
  • 支持激活函数复用,实现近无损精度下的高效非线性计算。
  • 创新缓冲管理策略,显著提升数据重用率,适合部署Mamba模型。

本文提出一种可重构架构的Mamba加速器MARCA。首先,设计了适用于线性与逐元素操作的可重构处理单元(PE)阵列:线性操作时启用归约树完成累加,逐元素操作时禁用归约树并直接输出。其次,基于可重构PE构建可复用的非线性函数单元:通过快速偏置指数算法将指数函数分解为逐元素操作与移位操作,利用分段逼近算法将SiLU激活函数拆解为范围检测与逐元素操作,实现非线性函数的高复用率计算且精度损失极小。第三,提出操作内与操作间两级缓冲管理策略:操作内策略最大化线性操作内部输入数据共享,操作间策略优化逐元素操作间的中间数据重用。在不同规模的Mamba模型上进行广泛实验,MARCA相较Intel Xeon 8358P CPU与NVIDIA Tesla A100 GPU分别实现最高463.22×/11.66×的加速比和最高9761.42×/242.52×的能效提升。

原文摘要 · Abstract (English)

We propose a Mamba accelerator with reconfigurable architecture, MARCA.We propose three novel approaches in this paper. (1) Reduction alternative PE array architecture for both linear and element-wise operations. For linear operations, the reduction tree connected to PE arrays is enabled and executes the reduction operation. For element-wise operations, the reduction tree is disabled and the output bypasses. (2) Reusable nonlinear function unit based on the reconfigurable PE. We decompose the exponential function into element-wise operations and a shift operation by a fast biased exponential algorithm, and the activation function (SiLU) into a range detection and element-wise operations by a piecewise approximation algorithm. Thus, the reconfigurable PEs are reused to execute nonlinear functions with negligible accuracy loss.(3) Intra-operation and inter-operation buffer management strategy. We propose intra-operation buffer management strategy to maximize input data sharing for linear operations within operations, and inter-operation strategy for element-wise operations between operations. We conduct extensive experiments on Mamba model families with different sizes.MARCA achieves up to 463.22$\times$/11.66$\times$ speedup and up to 9761.42$\times$/242.52$\times$ energy efficiency compared to Intel Xeon 8358P CPU and NVIDIA Tesla A100 GPU implementations, respectively.

Mamba加速器可重构能效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。