量化让大模型更小更快，适合在低算力设备部署
A Comprehensive Study on Quantization Techniques for Large Language Models
- 用降低数值精度的方法压缩大模型，减少存储和计算开销
- 1750亿参数模型可从350GB压缩，适配物联网等资源受限设备
- 系统梳理了量化技术原理与主流方法，适合研究者快速入门
自Transformer模型兴起以来，大型语言模型（LLMs）在学术界和工业界得到广泛应用，展现出卓越的AI性能。然而，其计算需求巨大，能源消耗高。例如，具有1750亿参数、存储需求达350GB的GPT-3等流行模型，在资源受限的物联网设备和嵌入式系统中部署面临严峻挑战，这些系统通常缺乏处理如此大模型的算力。量化通过将模型数值精度降至更小的离散值集合，成为降低模型大小并加速推理的有前景解决方案。本文对机器学习领域中的量化技术进行了全面分析，重点聚焦其在大型语言模型中的应用。首先探讨了量化的数学理论，随后回顾了常见量化方法及其实施方式。此外，还详细分析了几种应用于大型语言模型的主流量化方法，阐述其算法设计与性能表现。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have been extensively researched and used in both academia and industry since the rise in popularity of the Transformer model, which demonstrates excellent performance in AI. However, the computational demands of LLMs are immense, and the energy resources required to run them are often limited. For instance, popular models like GPT-3, with 175 billion parameters and a storage requirement of 350 GB, present significant challenges for deployment on resource-constrained IoT devices and embedded systems. These systems often lack the computational capacity to handle such large models. Quantization, a technique that reduces the precision of model values to a smaller set of discrete values, offers a promising solution by reducing the size of LLMs and accelerating inference. In this research, we provide a comprehensive analysis of quantization techniques within the machine learning field, with a particular focus on their application to LLMs. We begin by exploring the mathematical theory of quantization, followed by a review of common quantization methods and how they are implemented. Furthermore, we examine several prominent quantization methods applied to LLMs, detailing their algorithms and performance outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。