系统评估大模型量化在不同任务中的表现，发现4比特量化对小模型伤害大，但大模型更稳。
Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant
- 覆盖1B到405B参数模型，测试4种量化方法在13个数据集上的表现
- 70B以上模型在4比特量化下仍保持稳定，小模型易严重失准
- 硬任务不必然损失最大，量化会放大模型本身弱点，适合模型优化者参考
量化被视为低成本部署大中小语言模型的有前景方案。然而，以往研究多局限于困惑度或基础知识任务，缺乏对Llama-3等新模型的全面评估。本文对1B至405B参数的指令微调模型进行了全面评估，采用四种量化方法在13个数据集上测试。结果表明：(1) 量化模型通常优于小型FP16基线，但在指令遵循和幻觉检测上表现不佳；(2) FP8在各类任务中最为稳健，而权重仅量化中AWQ普遍优于GPTQ；(3) 小模型在4比特量化下出现严重精度下降，70B规模模型则保持性能稳定；(4) 难度高的任务未必带来最大精度损失，说明量化会放大模型内在缺陷而非简单随任务难度上升；(5) 基于LLM的评测（MT-Bench）显示编码与STEM任务显著退化，推理任务偶有提升。
原文摘要 · Abstract (English)
Quantization has gained attention as a promising solution for the cost-effective deployment of large and small language models. However, most prior work has been limited to perplexity or basic knowledge tasks and lacks a comprehensive evaluation of recent models like Llama-3.3. In this paper, we conduct a comprehensive evaluation of instruction-tuned models spanning 1B to 405B parameters, applying four quantization methods across 13 datasets. Our findings reveal that (1) quantized models generally surpass smaller FP16 baselines, yet they often struggle with instruction-following and hallucination detection; (2) FP8 consistently emerges as the most robust option across tasks, and AWQ tends to outperform GPTQ in weight-only quantization; (3) smaller models can suffer severe accuracy drops at 4-bit quantization, while 70B-scale models maintain stable performance; (4) notably, \textit{hard} tasks do not always experience the largest accuracy losses, indicating that quantization magnifies a model's inherent weaknesses rather than simply correlating with task difficulty; and (5) an LLM-based judge (MT-Bench) highlights significant performance declines in Coding and STEM tasks, though it occasionally reports improvements in reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。