针对大模型量化精度不足，提出分层敏感度优化方法。
Towards Superior Quantization Accuracy: A Layer-sensitive Approach
- 根据激活敏感度和权重峰度识别难量化层
- 在仅增加2%内存下，使困惑度降低9%
- 适合需要高精度部署的大模型开发者
大型视觉与语言模型在自然语言理解、问题求解、逻辑推理和知识检索等任务中展现出类人智能。然而，训练和部署这些模型需要大量计算资源，成为其广泛应用和进一步研究的重大障碍。为缓解此问题，已开发多种模型压缩技术以降低计算需求。然而，现有方法通常采用统一的量化配置，未能考虑不同层在量化过程中的难度差异。本文通过利用层敏感度特征（如激活敏感度和权重分布峰度），识别难以准确量化的层，并分配额外的内存预算。提出的SensiBoost和KurtBoost方法显著提升了量化精度，在LLama模型上相比基线实现最高9%的困惑度降低，且仅增加2%的内存开销。
原文摘要 · Abstract (English)
Large Vision and Language Models have exhibited remarkable human-like intelligence in tasks such as natural language comprehension, problem-solving, logical reasoning, and knowledge retrieval. However, training and serving these models require substantial computational resources, posing a significant barrier to their widespread application and further research. To mitigate this challenge, various model compression techniques have been developed to reduce computational requirements. Nevertheless, existing methods often employ uniform quantization configurations, failing to account for the varying difficulties across different layers in quantizing large neural network models. This paper tackles this issue by leveraging layer-sensitivity features, such as activation sensitivity and weight distribution Kurtosis, to identify layers that are challenging to quantize accurately and allocate additional memory budget. The proposed methods, named SensiBoost and KurtBoost, respectively, demonstrate notable improvement in quantization accuracy, achieving up to 9% lower perplexity with only a 2% increase in memory budget on LLama models compared to the baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。