通过端到端损失引导，提升大模型量化精度与性能。
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance
- 利用最终损失梯度指导量化过程，捕捉特征重要性差异。
- 在多种量化方式下均显著提升模型性能，优于当前最佳方法。
- 提出新型非均匀标量量化算法，收敛性有保证且效果更优。
后训练量化是通过不重新训练即可压缩大语言模型权重和激活值，从而降低内存占用和推理延迟的关键技术。然而，现有方法要么未能考虑隐藏特征对最终损失的不同重要性，要么在引入端到端损失时忽略了模型权重间的关键交互关系。为此，我们提出GuidedQuant，一种将最终损失梯度信息融入量化目标的新方法，同时保留输出通道内权重间的跨权重依赖。GuidedQuant在仅量化权重的标量、仅量化权重的向量以及权重与激活联合量化等多种场景中，持续提升当前最优量化方法的性能。此外，我们设计了一种新型非均匀标量量化算法，可保证量化目标值单调下降，并在该类别中表现更优。代码已开源：https://github.com/snu-mllab/GuidedQuant。
原文摘要 · Abstract (English)
Post-training quantization is a key technique for reducing the memory and inference latency of large language models by quantizing weights and activations without requiring retraining. However, existing methods either (1) fail to account for the varying importance of hidden features to the end loss or, when incorporating end loss, (2) neglect the critical interactions between model weights. To address these limitations, we propose GuidedQuant, a novel quantization approach that integrates gradient information from the end loss into the quantization objective while preserving cross-weight dependencies within output channels. GuidedQuant consistently boosts the performance of state-of-the-art quantization methods across weight-only scalar, weight-only vector, and weight-and-activation quantization. Additionally, we introduce a novel non-uniform scalar quantization algorithm, which is guaranteed to monotonically decrease the quantization objective value, and outperforms existing methods in this category. We release the code at https://github.com/snu-mllab/GuidedQuant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。