2-bit推理会引发重复循环等问题,通过轻量级控制可恢复准确率并实现真提速。
Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery

- 用高精度规划和循环救援,让2比特模型生成更稳定的推理路径。
- 在MATH-500上,Qwen3-8B准确率从17.2%提升至74.2%,Qwen3-32B从65.0%到87.2%。
- 适合关注低比特推理效率与可靠性、想落地部署的开发者。
大语言推理模型依赖长推理轨迹,导致推理成本高。尽管低比特量化能降低每词解码开销,我们发现激进的2比特推理无法实现端到端加速,因生成过程不稳定导致总词数暴涨。2比特量化常引发重复循环、预算耗尽、承诺延迟和推理片段未闭合等过程失败,而非仅降低答案准确率。我们分析了Qwen3模型在数学与常识基准上的完整推理轨迹,发现准确率下降与这些过程性故障紧密相关。为此,提出两种轻量级控制:FP16规划,为2比特模型提供短期高精度轮廓;循环救援,检测重复轨迹后或采纳早期答案,或回退至FP16。在MATH-500上,循环救援使Qwen3-8B准确率从17.2%升至74.2%,规划+循环救援使Qwen3-32B从65.0%升至87.2%。结果表明,当将极端低比特推理的失败视为可控制的生成病理时,结合轻量检测与选择性FP16支持,2比特推理既能恢复准确率,又能保持真实端到端速度优势。代码已开源。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit inference can fail to deliver end-to-end speedup because instability in the generation process inflates total token count. Instead of merely lowering answer accuracy, 2-bit quantization often produces much longer traces with repetitive loops, budget exhaustion, delayed commitment, and unclosed reasoning segments. We analyze full reasoning traces of Qwen3 reasoning models across mathematical and commonsense benchmarks and show that accuracy degradation is tightly linked to these process-level failures. To address them, we introduce two lightweight controls: FP16 planning, which gives the 2-bit model a short high-precision outline, and loop rescue, which detects repetitive traces and either commits to an earlier answer or falls back to FP16. On MATH-500, loop rescue improves Qwen3-8B accuracy from 17.2% to 74.2%, while planning plus loop rescue improves Qwen3-32B from 65.0% to 87.2%. Overall, our results show that extreme low-bit reasoning becomes practical when its failures are treated as controllable generation pathologies: with lightweight detection and selective FP16 support, 2-bit inference can recover accuracy while preserving real end-to-end speed. Our code is available at: https://github.com/brain-lab-research/quantized-reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。