提出无需调参的量化方法,让模型在低精度下仍保持高精度表现。
Error Diffusion: Post Training Quantization with Block-Scaled Number Formats for Neural Networks
- 将网络视为复合函数,逐层扩散量化误差以提升精度。
- 在视觉模型和大语言模型上均实现与主流方法相当的性能。
- 支持块缩放格式,适合追求高效部署的工程师使用。
量化可降低模型的硬件开销,如数据搬运、存储和乘加运算成本,但也会因参数精度下降而影响输出质量。为此,需开发能保持模型行为一致性的量化方法。块缩放数值格式等非传统编码方式,在固定比特预算下展现优势。本文提出误差扩散(Error Diffusion, ED),一种无需超参数、支持块缩放格式的后训练量化方法,不依赖反向传播或海森矩阵信息。通过将神经网络视为复合函数,逐层扩散量化误差,从而改善量化过程。同时,我们推出TensorCast——基于PyTorch的开源库,用于模拟包括块缩放在内的多种数值格式,助力神经网络量化研究。在多种架构(含视觉模型与大语言模型)上的实验表明,该方法持续取得竞争力结果。实验证实,块缩放格式是后训练量化中稳健有效的选择,有助于提升先进神经网络的实际部署效率。
原文摘要 · Abstract (English)
Quantization reduces the model's hardware costs, such as data movement, storage, and operations like multiply and addition. It also affects the model's behavior by degrading the output quality. Therefore, there is a need for methods that preserve the model's behavior when quantizing model parameters. More exotic numerical encodings, such as block-scaled number formats, have shown advantages for utilizing a fixed bit budget to encode model parameters. This paper presents error diffusion (ED), a hyperparameter-free method for post-training quantization with support for block-scaled data formats. Our approach does not rely on backpropagation or Hessian information. We describe how to improve the quantization process by viewing the neural model as a composite function and diffusing the quantization error in every layer. In addition, we introduce TensorCast, an open-source library based on PyTorch to emulate a variety of number formats, including the block-scaled ones, to aid the research in neural model quantization. We demonstrate the efficacy of our algorithm through rigorous testing on various architectures, including vision and large language models (LLMs), where it consistently delivers competitive results. Our experiments confirm that block-scaled data formats provide a robust choice for post-training quantization and could be used effectively to enhance the practical deployment of advanced neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。