量化让模型过度思考却答错，加个惩罚项就能解决
Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
- 用轻量惩罚项抑制过度思考标记，减少推理长度
- 量化后52%错误源于中途已得正确答案但未输出
- 不需训练，对1.5B到32B模型都有效，适合部署优化
后训练量化（PTQ）广泛用于高效部署大语言模型，但其对推理模型的影响尚不明确。在数学、编程和科学问答任务中，我们发现激进量化会降低准确率并增加思维链（CoT）长度。令人意外的是，在高达52%的量化模型失败案例中，模型已在中间步骤得出正确答案，但未将其作为最终答案输出。为探究原因，我们测量了量化与全精度输出分布之间的词元级KL散度。高KL散度位置与高下一词元熵强相关，且在此类位置，量化模型更频繁采样如'wait'、'but'、'alternatively'等过度思考标记。通过仅对一组精选的过度思考标记施加无需训练的逻辑值惩罚，可在5个基准测试上将推理链长度减少12%至23%，同时保持或提升准确率，覆盖1.5B至32B参数的5个模型及3种量化方法，实现准确率与推理成本间的有利帕累托前沿。相比惩罚其他词集，该方法显著减少量化模型产生的过度思考错误，最高降幅达58%。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) is widely used to deploy large language models efficiently, but its effect on reasoning models is not well understood. Across math, coding, and science QA, we find that aggressive PTQ reduces accuracy while increasing chain-of-thought (CoT) length. Surprisingly, we show that in up to 52% of the quantized models' failures, models reach the right answer in intermediate reasoning steps but do not output it as a final answer. To understand why quantization leads to this increase in overthinking errors, we measure the token-level KL divergence between quantized and full-precision output distributions. Positions with high KL divergence correlate strongly with high next-token entropy, and at these positions quantized models disproportionately sample overthinking markers such as "wait", "but", and "alternatively". We show that simply introducing a training-free logit penalty on a curated set of overthinking markers can reduce CoT length by 12--23% while preserving or improving accuracy across 5 models (1.5B-32B parameters), 3 quantization methods, and 5 benchmarks, yielding a favorable Pareto frontier of accuracy against reasoning cost compared to penalizing other token sets. Overthinking errors produced by quantized models are particularly reduced by up to 58%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。