MIVE用统一架构加速softmax、layernorm和rmsnorm，提升硬件效率。
MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration
- 设计统一向量引擎，共享计算路径实现三类归一化操作
- 物理芯片实测面积与能效优于主流专用加速器
- 适合追求低延迟高能效的LLM推理硬件开发
大型语言模型（LLMs）的快速发展对满足严格推理延迟和功耗约束的专用硬件加速器提出了更高要求。尽管矩阵乘法占据主要计算负载，但LayerNorm、RMSNorm和Softmax等非线性向量归一化操作可能成为关键硬件瓶颈。现有加速器通常采用独立硬件模块实现这些功能，导致资源重复和硅片利用率低下。为此，本文提出极简整数向量引擎（MIVE），一种可编程架构，能在统一数据通路中执行上述三种操作。通过挖掘LayerNorm、RMSNorm和Softmax之间的共性计算模式，该向量引擎最大化硬件共享，降低实现开销。物理ASIC实现结果表明，MIVE在提供全功能支持的同时，相比多数最先进独立加速器，实现了更高的面积效率和硬件效率。
原文摘要 · Abstract (English)
The rapid growth of Large Language Models (LLMs) has intensified the need for specialized hardware accelerators that can satisfy stringent inference latency and power constraints. Although matrix multiplications dominate the overall computational workload, non-linear vector normalization operations, such as LayerNorm, RMSNorm and Softmax can become critical hardware bottlenecks. Existing accelerators typically implement these functions using dedicated hardware blocks, leading to duplicated resources and inefficient silicon utilization. To address this limitation, we propose a Minimalist Integer Vector Engine (MIVE), a programmable architecture capable of executing all three operations within a unified datapath. By exploiting common computational patterns across LayerNorm, RMSNorm and Softmax the proposed vector engine maximizes hardware sharing while reducing implementation overhead. Physical ASIC implementation results show that MIVE provides comprehensive multi-function support while achieving higher area and hardware efficiency than most state-of-the-art standalone accelerators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。