低比特量化训练让大模型推理更高效，关键在知识蒸馏与初始化优化。
What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
- 用知识蒸馏+预训练量化初始化，提升低比特量化效果。
- 2比特下性能恢复至接近原始模型，数学任务提升44.53%。
- 适合想高效部署推理大模型的研究者和工程师。
推理模型在编码和数学等复杂任务中表现优异，但推理速度慢且 token 效率低。为提升推理效率，后训练量化（PTQ）常导致准确率大幅下降，尤其在低比特设置下。本文对推理模型的量化感知训练（QAT）进行系统性实证研究，发现：(1) 知识蒸馏是监督微调或强化学习训练模型的鲁棒目标；(2) PTQ 提供强初始化，提升准确率并降低训练成本；(3) 在可行冷启动条件下，强化学习仍适用于量化模型并带来额外增益；(4) 将 PTQ 校准域与 QAT 训练域对齐可加速收敛并提高最终准确率。最后，我们整合这些发现构建了优化流程 Reasoning-QAT，其在多个 LLM 骨干和推理数据集上持续优于现有 SOTA PTQ 方法。例如，在 Qwen3-0.6B 上，于 MATH-500 数据集上比 GPTQ 提升 44.53%，并在 2 比特设置下稳定恢复性能。
原文摘要 · Abstract (English)
Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To improve the inference efficiency, post-training quantization (PTQ) usually comes with the cost of large accuracy drops, especially for reasoning tasks under low-bit settings. In this study, we present a systematic empirical study of quantization-aware training (QAT) for reasoning models. Our key findings include: (1) Knowledge distillation is a robust objective for reasoning models trained via either supervised fine-tuning or reinforcement learning; (2) PTQ provides a strong initialization for QAT, improving accuracy while reducing training cost; (3) Reinforcement learning remains feasible for quantized models given a viable cold start and yields additional gains; and (4) Aligning the PTQ calibration domain with the QAT training domain accelerates convergence and often improves the final accuracy. Finally, we consolidate these findings into an optimized workflow (Reasoning-QAT), and show that it consistently outperforms state-of-the-art PTQ methods across multiple LLM backbones and reasoning datasets. For instance, on Qwen3-0.6B, it surpasses GPTQ by 44.53% on MATH-500 and consistently recovers performance in the 2-bit regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。