通过合作博弈优化大模型分层量化,2比特下仍保持高精度。
CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs
- 将各层量化敏感度建模为合作博弈,捕捉层间交互影响。
- 在4比特至2比特平均精度下,困惑度降低20%至80%。
- 适用于低资源部署场景,尤其适合追求极致压缩的开发者。
大型语言模型虽具强大能力,但其数十亿参数规模使设备端或低资源部署变得困难。混合精度量化提供有效解决方案,但现有方法在平均精度低于4比特时表现不佳,因其依赖孤立的层级指标,忽视了影响整体性能的关键层间交互。为此,我们首次将混合精度量化问题建模为层间的合作博弈,提出基于沙普利值的渐进式量化估计(SPQE),高效获得层敏感度与层间交互的准确估计。利用SPQE结果,我们设计了合作博弈启发的混合精度量化方法(CoopQ),将沙普利值转化为二元二次优化问题,严格在内存约束下为各层分配2比特或4比特精度。在Llama-3、Gemma-2和Qwen-3模型上,结合三个独立的后训练量化后端(Quanto、HQQ、GPTQ)进行的全面实验表明,CoopQ具有优异可扩展性,性能始终优于仅依赖孤立指标的方法。在平均精度从4比特降至2比特的范围内,相较最优基线,困惑度相对下降20%至80%,且压缩越紧,优势越明显。
原文摘要 · Abstract (English)
Large Language Models (LLMs) promise impressive capabilities, yet their multi-billion-parameter scale makes on-device or low-resource deployment prohibitive. Mixed-precision quantization offers a compelling solution, but existing methods struggle when the average precision drops below four bits, as they rely on isolated, layer-specific metrics that overlook critical inter-layer interactions affecting overall performance. To address these limitations, we first frame the mixed-precision quantization problem as a cooperative game among layers and introduce Shapley-based Progressive Quantization Estimation (SPQE) to efficiently obtain accurate Shapley estimates of layer sensitivities and inter-layer interactions. Leveraging the SPQE estimates, we propose Cooperative Game Inspired Mixed-Precision Quantization (CoopQ) which translates these Shapley estimates into a binary quadratic optimization formulation, assigning either 2 or 4-bit precision to layers under strict memory constraints. Comprehensive experiments conducted on Llama-3, Gemma-2, and Qwen-3 models across three independent PTQ backends (Quanto, HQQ, GPTQ) demonstrate CoopQ's scalability and consistently superior performance compared to methods relying solely on isolated metrics. Across average precisions spanning 4 bit down to 2 bit, CoopQ cuts Perplexity by 20 - 80 % relative to the best baseline, with the margin growing as the bit-width tightens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。