提出新方法TASA,让模型量化更精准高效。
Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization

- 基于任务感知分析,优化校准数据组合与比特分配。
- 3.5比特模型性能超越4比特基线,关键任务提升超20点。
- 适合需要高效部署大模型的开发者和研究者。
混合精度量化(MPQ)是应对内存与计算约束部署大语言模型的关键技术。我们首次发现‘困惑度幻觉’现象:基于困惑度敏感性排序的重要层,与复杂推理性能贡献高的层排名相关性极低(Kendall τ≈0)。进一步揭示‘对齐-多样性权衡’:仅用目标任务校准数据会损害量化后性能,加入通用领域数据可稳定敏感性估计并提升跨任务鲁棒性。为此提出TASA(任务感知敏感性分析),一个两级框架,联合优化校准数据组成与混合精度比特分配。TASA采用无训练梯度迹对齐准则搜索校准数据混合,并融合困惑度与推理导向敏感信号,指导层间与层内比特分配。在LLaMA-3-8B和Qwen2.5-7B上的实验显示,合理分配的3.5比特模型可媲美或超越非任务感知的4比特基线。平均精度3.5比特时,TASA在综合准确率上匹配或超过多个竞争性4比特均匀基线,在LLaMA-3-8B上于GSM8K任务上优于最强的W3基线超过20绝对百分点。结果表明,校准数据构成显著影响任务敏感量化,这一因素此前被严重低估。
原文摘要 · Abstract (English)
Mixed-precision quantization (MPQ) has become a key technique for deploying large language models under stringent memory and compute constraints. We first identify a phenomenon that we term the Perplexity Illusion: layers ranked as important by perplexity-based sensitivity show little rank correlation with those that are most influential for complex reasoning performance, with Kendall $τ\approx 0$ in our analysis. We further reveal an Alignment-Diversity Tradeoff: using only target-task calibration data can degrade post-quantization performance, whereas incorporating general-domain data stabilizes sensitivity estimation and improves robustness across tasks. Based on these observations, we propose TASA (Task-Aware Sensitivity Analysis), a two-level framework that jointly optimizes calibration-data composition and mixed-precision bit allocation. Specifically, TASA searches for a calibration-data mixture using a training-free gradient-trace alignment criterion, and then aggregates perplexity and reasoning-oriented sensitivity signals to guide both inter-layer and intra-layer bit allocation. Experiments on LLaMA-3-8B and Qwen2.5-7B reveal a precision inversion: appropriately allocated 3.5-bit models can match or surpass less task-aware 4-bit baselines. At an average precision of 3.5 bits, TASA matches or outperforms several competitive 4-bit uniform baselines in aggregate accuracy, and improves over the strongest W3 baseline on GSM8K by more than 20 absolute points on LLaMA-3-8B. These results show that calibration-data composition substantially affects task-sensitive quantization, a factor underexplored in prior work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。