首次系统研究量化对推理模型的影响,发现低比特量化会显著降低精度。
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models

- 测试了多种量化方法与比特位宽,覆盖大模型到小模型
- W8A8或W4A16可实现无损量化,但更低比特会严重损害性能
- 模型规模、来源和任务难度是影响表现的关键因素
近期推理型语言模型在复杂任务中表现出色,但其长链式思维过程增加了推理开销。尽管量化已被广泛用于降低大模型推理成本,但其对推理模型的影响仍缺乏系统研究。本文首次系统评估了从1.5B到70B参数的DeepSeek-R1-Distilled Qwen、LLaMA系列,以及QwQ-32B和Qwen3-8B等模型,采用最先进的算法,在不同比特位宽下进行权重、键值缓存和激活量化,并在数学(AIME、MATH-500)、科学(GPQA)和编程(LiveCodeBench)推理基准上展开全面评估。结果表明,使用W8A8或W4A16可实现无损量化,但更低比特位宽会带来显著精度损失。我们进一步发现模型大小、来源和任务难度是决定性能的关键因素。出乎意料的是,量化模型并未出现输出长度增加的现象。此外,通过合理扩展模型规模或推理步数可有效提升性能。所有量化模型与代码已开源至https://github.com/ruikangliu/Quantized-Reasoning-Models。
原文摘要 · Abstract (English)
Recent advancements in reasoning language models have demonstrated remarkable performance in complex tasks, but their extended chain-of-thought reasoning process increases inference overhead. While quantization has been widely adopted to reduce the inference cost of large language models, its impact on reasoning models remains understudied. In this paper, we conduct the first systematic study on quantized reasoning models, evaluating the open-sourced DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B parameters, QwQ-32B, and Qwen3-8B. Our investigation covers weight, KV cache, and activation quantization using state-of-the-art algorithms at varying bit-widths, with extensive evaluation across mathematical (AIME, MATH-500), scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our findings reveal that while lossless quantization can be achieved with W8A8 or W4A16 quantization, lower bit-widths introduce significant accuracy risks. We further identify model size, model origin, and task difficulty as critical determinants of performance. Contrary to expectations, quantized models do not exhibit increased output lengths. In addition, strategically scaling the model sizes or reasoning steps can effectively enhance the performance. All quantized models and codes are open-sourced in https://github.com/ruikangliu/Quantized-Reasoning-Models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。