CPTQuant通过分层敏感度分析，实现大模型高效低精度压缩。
CPTQuant - A Novel Mixed Precision Post-Training Quantization Techniques for Large Language Models
- 基于相关性、剪枝和泰勒分解，动态分配不同层的精度。
- 最高实现4倍压缩、2倍效率提升，精度损失极小。
- 适合部署大模型的开发者，尤其关注推理加速与存储优化。
大型语言模型虽显著提升自然语言理解与生成能力，但其内存与计算开销巨大。量化技术成为缓解此问题的有力手段，可在保持精度的同时提升能效。本文提出CPTQuant，一种综合策略，包含基于相关性的混合精度量化（CMPQ）、基于剪枝的（PMPQ）和基于泰勒分解的（TDMPQ）方法。CMPQ根据各层的典型相关性调整精度；PMPQ依据层对稀疏性的敏感度进行逐层优化；TDMPQ通过泰勒分解评估层对输入扰动的敏感度以调节精度。该策略将更高精度分配给敏感层，降低鲁棒层的精度。在BERT、OPT-125M、OPT-350M、OPT-1.3B和OPT-2.7B上验证性能，实现最高4倍压缩与2倍效率提升，相比Hugging Face FP16精度损失极小。敏感度分析表明，初始与末尾30%的层更敏感。在分类任务中，PMPQ比其他方法压缩率高11%；在语言建模任务中，TDMPQ压缩率高出30%。
原文摘要 · Abstract (English)
Large language models have transformed the comprehension and generation of natural language tasks, but they come with substantial memory and computational requirements. Quantization techniques have emerged as a promising avenue for addressing these challenges while preserving accuracy and making energy efficient. We propose CPTQuant, a comprehensive strategy that introduces correlation-based (CMPQ), pruning-based (PMPQ), and Taylor decomposition-based (TDMPQ) mixed precision techniques. CMPQ adapts the precision level based on canonical correlation analysis of different layers. PMPQ optimizes precision layer-wise based on their sensitivity to sparsity. TDMPQ modifies precision using Taylor decomposition to assess each layer's sensitivity to input perturbation. These strategies allocate higher precision to more sensitive layers while diminishing precision to robust layers. CPTQuant assesses the performance across BERT, OPT-125M, OPT-350M, OPT-1.3B, and OPT-2.7B. We demonstrate up to 4x compression and a 2x-fold increase in efficiency with minimal accuracy drop compared to Hugging Face FP16. PMPQ stands out for achieving a considerably higher model compression. Sensitivity analyses across various LLMs show that the initial and final 30% of layers exhibit higher sensitivities than the remaining layers. PMPQ demonstrates an 11% higher compression ratio than other methods for classification tasks, while TDMPQ achieves a 30% greater compression ratio for language modeling tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。