量化对大模型影响有限，可放心用于资源受限部署
Interpreting the Effects of Quantization on LLMs
- 用可解释性方法分析4比特和8比特量化的影响
- 死神经元数量和模型校准基本不变
- 小模型更少关键神经元，大模型更冗余
量化为在资源受限环境下部署大语言模型提供了实用方案。然而，其对内部表征的影响仍研究不足，引发对量化模型可靠性的质疑。本研究采用多种可解释性技术，考察4比特和8比特量化对多个大语言模型的模型行为与神经元特性的影响。结果表明，量化对模型校准的影响总体较小。神经元激活分析显示，死神经元（在整个数据集上激活值趋近于0的神经元）数量在量化前后保持一致。在神经元对预测的贡献方面，较小的全精度模型表现出较少显著神经元，而较大的模型则有更多，但Llama-2-7B例外。量化对神经元冗余的影响因模型而异。总体而言，量化影响因模型和任务而异，但未发现剧烈变化，支持量化作为可靠的模型压缩技术。
原文摘要 · Abstract (English)
Quantization offers a practical solution to deploy LLMs in resource-constraint environments. However, its impact on internal representations remains understudied, raising questions about the reliability of quantized models. In this study, we employ a range of interpretability techniques to investigate how quantization affects model and neuron behavior. We analyze multiple LLMs under 4-bit and 8-bit quantization. Our findings reveal that the impact of quantization on model calibration is generally minor. Analysis of neuron activations indicates that the number of dead neurons, i.e., those with activation values close to 0 across the dataset, remains consistent regardless of quantization. In terms of neuron contribution to predictions, we observe that smaller full precision models exhibit fewer salient neurons, whereas larger models tend to have more, with the exception of Llama-2-7B. The effect of quantization on neuron redundancy varies across models. Overall, our findings suggest that effect of quantization may vary by model and tasks, however, we did not observe any drastic change which may discourage the use of quantization as a reliable model compression technique.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。