按层智能选量化方法，让大模型更省显存还更准
CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization
- 用CKA度量各层特性，自动匹配最优量化算法
- 在LLaMA和Qwen上比统一量化降低12%困惑度
- 无需微调，可直接插入现有训练流程使用
当前主流的大语言模型后训练量化方法通常对所有网络层采用统一的量化策略，忽略了各层在算法适配性上的显著差异。为解决这一问题，我们提出CALM（一种基于CKA的自适应分层模块化框架），一个无需微调、即插即用的异构量化框架。CALM独立评估多个后训练量化算法在每一层的表现，并利用线性中心核对齐（CKA）作为指标，自动选择每层最优的量化策略。将各层优化后的策略整合，构建混合量化模型。实验表明，该方法在包括LLaMA和Qwen在内的主流大模型上，均一致优于统一量化基线和最先进的混合精度方法，在困惑度（PPL）和下游任务性能上表现更优。
原文摘要 · Abstract (English)
Current mainstream post-training quantization methods for large language models typically apply a uniform quantization strategy across all network layers, overlooking the substantial differences in algorithmic suitability among layers. To address this limitation, we propose CALM (A CKA-guided Adaptive Layer-wise Modularization)a fine-tuning-free, plug-and-play framework for algorithmic heterogeneous quantization. CALM independently evaluates multiple PTQ algorithms on each layer and employs Linear Centered Kernel Alignment (CKA) as a metric to automatically select the optimal quantization strategy per layer. The individually optimized strategies are then integrated to construct a hybrid quantized model. Experiments demonstrate that our approach consistently outperforms both uniform quantization baselines and state-of-the-art mixed-precision methods across mainstream LLMsincluding LLaMA and Qwenin terms of perplexity (PPL) and downstream task performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。