SISA通过分块矩阵计算，让加速器在处理大模型时更高效节能。
SISA: A Scale-In Systolic Array for GEMM Acceleration
- 将传统方形阵列拆成水平矩形区块，灵活调度小矩阵运算
- 相比同类芯片，速度提升8.52倍，能效降低93%
- 适合大模型中不规则数据的算力需求，硬件改造成本低
当前主流的AI/ML工作负载，如大语言模型（LLMs），依赖高效的通用矩阵乘法（GEMM）运算。因此，多数系统采用基于处理单元（PEs）的正方形脉动阵列（SAs）专用硬件加速器。尽管该结构对传统深度神经网络（DNNs）有效，但大语言模型引入了输入相关且高度倾斜的矩阵，导致脉动阵列资源利用率低下。为此，我们提出SISA（Scale-In Systolic Array），一种新型脉动阵列架构，将传统正方形阵列划分为水平矩形条带。在极小开销下，SISA通过独立调度条带暴露并行性，高效执行小规模或倾斜的矩阵运算，同时保留完整阵列操作以应对大型GEMM。与同等处理单元数量的先进单体脉动阵列相比，SISA在代表性大语言模型上实现最高8.52倍加速和93%的能量延迟积（EDP）降低。
原文摘要 · Abstract (English)
The currently dominant AI/ML workloads, such as Large Language Models (LLMs), rely on the efficient execution of General Matrix-Matrix Multiplication (GEMM) operations. Thus, most systems are equipped with dedicated matrix hardware accelerators based on square Systolic Arrays (SAs) of Processing Elements (PEs). While this organization was effective for traditional Deep Neural Networks (DNNs), LLMs introduce input-dependent and highly skewed matrices, leading to underutilized SA resources. To address this challenge, we propose SISA (Scale-In Systolic Array), a novel SA architecture that partitions the traditional square array into horizontal rectangular slabs. With minimal overhead, SISA exposes parallelism through independently scheduled slabs for efficient execution of small or skewed matrix shapes, while retaining full-array operation for large GEMMs. SISA achieves up to 8.52x speedup and 93% energy-delay-product (EDP) reduction for representative LLMs compared to a state-of-the-art monolithic SA with the same number of PEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。