为边缘设备设计高效且保证归一化的软最大与层归一化硬件
Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices
- 用硬件友好的近似方法保持归一化特性
- 面积仅942μm²（Softmax）和1199μm²（LayerNorm），面积减少达14倍
- 适用于边缘NLP与生成式AI等需精准分数的任务
在Transformer模型中，非GEMM操作（尤其是软最大与层归一化）因非线性特性常成为硬件成本主要来源。现有近似方法多针对秩相关任务，适用于分类任务，但边缘自然语言处理与生成式AI更依赖评分类任务，因此必须保证归一化。本文提出面向边缘设备的软最大与层归一化硬件高效实现方案，采用硬件友好近似方法，同时确保归一化特性（软最大：∑p=1，层归一化：σ=1）。设计使用Verilog HDL描述，并在三星28nm CMOS工艺下综合。精度评估显示性能损失极小：GLUE得分提升0.07%，SQuAD下降0.01%，困惑度下降0.09%。实现结果表明，软最大模块面积为942μm²，层归一化为1199μm²，相比当前最优方案分别减少11倍与14倍。
原文摘要 · Abstract (English)
In Transformer models, non-GEMM (non-General Matrix Multiplication) operations -- especially Softmax and Layer Normalization (LayerNorm) -- often dominate hardware cost due to their nonlinear nature. To address this, previous approximation studies mainly target rank-oriented tasks, which is acceptable for classification. However, edge Natural Language Processing (NLP) applications and edge generative AI are largely evaluated based on score-oriented tasks, so normalization-guaranteed non-GEMM operations are essential. We propose a hardware-efficient Softmax and LayerNorm with Guaranteed Normalization for Edge devices. Our design employs hardware-efficient approximation methods while preserving the normalization (Softmax: $\sum p = 1$, LayerNorm: $σ= 1$). Our architecture is described in Verilog HDL and synthesized using the Samsung 28nm CMOS process. In accuracy evaluation, we achieve high accuracy with minimal degradation: GLUE +0.07%, SQuAD -0.01%, perplexity -0.09%. Implementation results show that our architecture is small: $942\,μm^2$ for Softmax, $1199\,μm^2$ for LayerNorm. Compared to the state of the art, we achieve up to 11x and 14x reduction in area, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。