提出快速迭代归一化方法，显著降低大模型计算中的数据移动开销。
IterL2Norm: Fast Iterative L2-Normalization
- 设计迭代L2归一化算法，5步内收敛，精度高于传统方法。
- 在64-1024维向量上实现116-227周期延迟，频率100MHz。
- 适合部署于加速器芯片，提升Transformer推理效率。
基于Transformer的大语言模型是内存受限型模型，其运算依赖大量边际复用的数据，因此主机与加速器间的数据传输很可能决定总运行时间。层归一化是Transformer模型中关键计算负载之一，出现在每个多头注意力和前馈网络模块之后。为减少数据移动，层归一化需在矩阵乘法引擎所在同一芯片上执行。为此，我们提出一种针对1维输入的迭代L2归一化方法（IterL2Norm），可在五步迭代内快速收敛至稳态解，并保持高精度，在OPT模型使用的嵌入长度下，对FP32有六/九种情况优于快速平方根倒数算法，对BFloat16有五/九种情况更优。该方法采用32/28nm CMOS工艺实现，可对维度为$ d $（$64 \≤ d \≤ 1024$）的向量进行归一化，工作在100MHz/1.05V条件下，延迟为116-227周期。
原文摘要 · Abstract (English)
Transformer-based large language models are a memory-bound model whose operation is based on a large amount of data that are marginally reused. Thus, the data movement between a host and accelerator likely dictates the total wall-clock time. Layer normalization is one of the key workloads in the transformer model, following each of multi-head attention and feed-forward network blocks. To reduce data movement, layer normalization needs to be performed on the same chip as the matrix-matrix multiplication engine. To this end, we introduce an iterative L2-normalization method for 1D input (IterL2Norm), ensuring fast convergence to the steady-state solution within five iteration steps and high precision, outperforming the fast inverse square root algorithm in six out of nine cases for FP32 and five out of nine for BFloat16 across the embedding lengths used in the OPT models. Implemented in 32/28nm CMOS, the IterL2Norm macro normalizes $d$-dimensional vectors, where $64 \leq d \leq 1024$, with a latency of 116-227 cycles at 100MHz/1.05V.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。