MARCA通过可重构架构,大幅提升Mamba模型的推理速度与能效。
MARCA: Mamba Accelerator with ReConfigurable Architecture
- 采用可重构计算阵列,动态切换线性与逐元素运算路径。
- 支持激活函数复用,实现近无损精度下的高效非线性计算。
- 创新缓冲管理策略,显著提升数据重用率,适合部署Mamba模型。
本文提出一种可重构架构的Mamba加速器MARCA。首先,设计了适用于线性与逐元素操作的可重构处理单元(PE)阵列:线性操作时启用归约树完成累加,逐元素操作时禁用归约树并直接输出。其次,基于可重构PE构建可复用的非线性函数单元:通过快速偏置指数算法将指数函数分解为逐元素操作与移位操作,利用分段逼近算法将SiLU激活函数拆解为范围检测与逐元素操作,实现非线性函数的高复用率计算且精度损失极小。第三,提出操作内与操作间两级缓冲管理策略:操作内策略最大化线性操作内部输入数据共享,操作间策略优化逐元素操作间的中间数据重用。在不同规模的Mamba模型上进行广泛实验,MARCA相较Intel Xeon 8358P CPU与NVIDIA Tesla A100 GPU分别实现最高463.22×/11.66×的加速比和最高9761.42×/242.52×的能效提升。
原文摘要 · Abstract (English)
We propose a Mamba accelerator with reconfigurable architecture, MARCA.We propose three novel approaches in this paper. (1) Reduction alternative PE array architecture for both linear and element-wise operations. For linear operations, the reduction tree connected to PE arrays is enabled and executes the reduction operation. For element-wise operations, the reduction tree is disabled and the output bypasses. (2) Reusable nonlinear function unit based on the reconfigurable PE. We decompose the exponential function into element-wise operations and a shift operation by a fast biased exponential algorithm, and the activation function (SiLU) into a range detection and element-wise operations by a piecewise approximation algorithm. Thus, the reconfigurable PEs are reused to execute nonlinear functions with negligible accuracy loss.(3) Intra-operation and inter-operation buffer management strategy. We propose intra-operation buffer management strategy to maximize input data sharing for linear operations within operations, and inter-operation strategy for element-wise operations between operations. We conduct extensive experiments on Mamba model families with different sizes.MARCA achieves up to 463.22$\times$/11.66$\times$ speedup and up to 9761.42$\times$/242.52$\times$ energy efficiency compared to Intel Xeon 8358P CPU and NVIDIA Tesla A100 GPU implementations, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。