对比了两种量化方法，显著缩小模型体积并提升边缘设备运行效率。
Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques
- 采用新提出的缩放因子γ，优化量化过程以保持性能
- INT8和INT4量化分别降低40%和60%计算开销与功耗
- 适用于需要在低资源设备部署大模型的开发者
本文系统分析了用于优化大语言模型（LLMs）的量化技术，重点比较后训练量化（PTQ）与量化感知训练（QAT）。在参数规模从10M到1B的多个模型上进行实验，结果表明，采用我们提出的缩放因子γ，量化可实现高达68%的模型尺寸压缩，同时性能仅比全精度基线下降6%以内。实验显示，INT8量化可降低40%的计算成本与功耗，而INT4量化进一步提升至60%。我们提出一种新的混合精度量化理论框架，基于层敏感性和权重方差推导出最优位数分配策略。在边缘设备上的硬件效率评估显示，该量化方案使INT8推理吞吐量提升2.4倍，INT4达3倍，功耗降低60%。
原文摘要 · Abstract (English)
This paper presents a comprehensive analysis of quantization techniques for optimizing Large Language Models (LLMs), specifically focusing on Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). Through empirical evaluation across models ranging from 10M to 1B parameters, we demonstrate that quantization can achieve up to 68% reduction in model size while maintaining performance within 6% of full-precision baselines when utilizing our proposed scaling factor γ. Our experiments show that INT8 quantization delivers a 40% reduction in computational cost and power consumption, while INT4 quantization further improves these metrics by 60%. We introduce a novel theoretical framework for mixed-precision quantization, deriving optimal bit allocation strategies based on layer sensitivity and weight variance. Hardware efficiency evaluations on edge devices reveal that our quantization approach enables up to 2.4x throughput improvement for INT8 and 3x for INT4, with 60% power reduction compared to full-precision models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。