4-bit量化推理模型在测试时动态扩增，解决误判提前停止问题。
BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
- 设计轻量级运行时控制器，结合不确定性和推理稳定性在线估算。
- 在512个词元上限下，7B和14B模型准确率分别提升3.7和2.8点。
- 无需微调模型，适配主流4比特推理框架，适合资源受限场景。
后训练量化使大模型在内存与延迟约束下具备实用性，但会扭曲驱动自适应测试时计算分配的在线信号。在新生成词元数固定的情况下，信心失准可能导致有害的过早终止：模型可能呈现看似合理的最终答案，而底层推理仍错误；或控制器在推理轨迹未稳定前就停止。本文针对贪心4比特推理研究该问题，提出BitCal-TTS——一种轻量级运行时控制器，包含（i）低成本的词元级不确定性与推理轨迹稳定性在线代理，（ii）低精度下保守的比特条件置信度重校准，（iii）面向GSM8K类结构化输出的比特感知后标记确认窗口。方法无需微调基模型，通过前向钩子集成至标准Hugging Face 4比特推理流程。在小规模GSM8K评估片段上（7B: N=54, 14B: N=35），BitCal-TTS在7B和14B模型上均优于非比特感知基线，准确率提升+3.7和+2.8点，过早停止率从14.8%降至11.1%（7B）、17.1%降至11.4%（14B）。全程报告威尔逊95%置信区间，并明确讨论部分片段比较的统计效力有限。代码与绘图脚本已开源以支持复现。
原文摘要 · Abstract (English)
Post-training quantization makes large reasoning models practical under tight memory and latency budgets, but it can distort the online signals that drive adaptive test-time compute allocation. Under a fixed cap on the number of newly generated tokens, miscalibrated confidence can lead to harmful early halting: the model may surface a plausible final line while the underlying reasoning is still wrong, or the controller may stop before the trace has stabilized. We study this interaction for greedy 4-bit inference and propose BitCal-TTS, a lightweight runtime controller that combines (i) inexpensive online proxies for token-level uncertainty and reasoning-trace stability, (ii) a bit-conditioned confidence rescaling that is conservative at low nominal precision, and (iii) a bit-aware post-marker confirmation horizon designed for GSM8K-style structured outputs. The method requires no fine-tuning of the base model and integrates with standard Hugging Face 4-bit inference using forward hooks for logits and last-layer hidden states. On small evaluation shards of GSM8K with Qwen2.5 Instruct models, BitCal-TTS improves exact-match accuracy over a non-bit-aware adaptive baseline at the 7B and 14B scales while preserving substantial token savings relative to fixed-budget decoding. At a token cap of B=512, on the evaluation shards we report (N=54 for 7B and N=35 for 14B; not the full GSM8K test set), accuracy gains are +3.7 points (7B) and +2.8 points (14B), with the premature-stop rate falling from 14.8% to 11.1% on 7B and from 17.1% to 11.4% on 14B. We report Wilson 95% confidence intervals throughout and explicitly discuss the limited statistical power of the partial-shard comparisons. We release code and figure-generation scripts to support full reproduction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。