Mugi通过值级并行提升大模型效率，显著加速推理并降低能耗。
Mugi: Value Level Parallelism For Efficient LLMs
- 基于重要值分配更高精度的值中心策略优化非线性计算
- 支持小批量异构输入，结合量化与注意力优化提升效率
- 适配全量大模型工作负载，适合追求高性能低功耗部署的场景
值级并行（VLP）被提出用于提升对称激活与权重间大批次、低精度矩阵乘法的效率。在基于Transformer的大语言模型中，存在超越激活-权重矩阵乘法的更复杂操作。本文探索了VLP在大语言模型中的应用：首先，将VLP推广至非线性近似，其端到端大模型准确率、性能和效率均优于现有方法；该近似采用值中心策略，为关键值分配更高精度。其次，针对小批次异构输入的GEMM高效优化，融合权重仅量化、键值缓存量化与分组查询注意力等及时优化技术。最后，设计新架构Mugi以集成上述创新，支持完整大模型工作负载，并实现更优性能、效率与可持续性。实验表明，Mugi在非线性Softmax操作上可实现最高45倍吞吐提升和668倍能效提升，在大模型整体上分别提升2.07倍和3.11倍，同时运营碳排放减少1.45倍，隐含碳排放减少1.48倍。
原文摘要 · Abstract (English)
Value level parallelism (VLP) has been proposed to improve the efficiency of large-batch, low-precision general matrix multiply (GEMM) between symmetric activations and weights. In transformer based large language models (LLMs), there exist more sophisticated operations beyond activation-weight GEMM. In this paper, we explore how VLP benefits LLMs. First, we generalize VLP for nonlinear approximations, outperforming existing nonlinear approximations in end-to-end LLM accuracy, performance, and efficiency. Our VLP approximation follows a value-centric approach, where important values are assigned with greater accuracy. Second, we optimize VLP for small-batch GEMMs with asymmetric inputs efficiently, which leverages timely LLM optimizations, including weight-only quantization, key-value (KV) cache quantization, and group query attention. Finally, we design a new VLP architecture, Mugi, to encapsulate the innovations above and support full LLM workloads, while providing better performance, efficiency and sustainability. Our experimental results show that Mugi can offer significant improvements on throughput and energy efficiency, up to $45\times$ and $668\times$ for nonlinear softmax operations, and $2.07\times$ and $3.11\times$ for LLMs, and also decrease operational carbon for LLM operation by $1.45\times$ and embodied carbon by $1.48\times$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。