arXiv:2607.11317cs.AIcs.IT2026-07

提出新方法检测量化模型推理失效,比传统方法更准且无需训练。

Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors

  • 用不确定度与重复检测融合的报警分数,识别推理异常。
  • 在GSM8K上将误报率降至30%以下,精准定位失败轨迹。
  • 适合部署低比特量化模型时做推理质量监控,尤其关注重复问题。

低比特量化使小模型部署成本降低,但会削弱其推理链质量。为此需在解码端设置监控机制,在生成不可靠时介入。我们发现,常见的中心化词元对数概率增量(“log p(w_t)+H_t”)不适合作为监控指标:根据模型采样分布,该量是零均值鞅,仅反映采样自洽性,而非轨迹健康状态,且在高置信度重复时几乎无声。本文提出一种无训练解码控制器,结合(i)融合词元不确定性与显式原文重复的退化感知报警分数,及(ii)受校准e-过程启发的序列检测器。原始乘积过程在条件均值零假设下满足Ville有效性,而部署中采用的带截断的CUSUM统计量作为经验变化检测器,因分数具历史依赖性和自相关性。在FP16与INT4下的DeepSeek-R1-Distill-Qwen-1.5B模型上,校准后监控器从93%-95%的触发率降为ϕ≈0.3、精确率≈0.6(基线0.38),显著提升针对性。初步实验显示,控制器降低了原文重复信号,使INT4精度从63%提升至69%(配对McNemar检验,p=0.18,n=100),代价为28%的词元预算增加。此外,非终止是GSM8K上的主要失败模式。核心贡献在于方法论:解释为何中心化对数概率不适用,并提供一个经校准的替代方案。

原文摘要 · Abstract (English)

Low-bit quantization makes small reasoning models inexpensive to deploy but can degrade their chains of thought. This motivates decoder-side monitors that intervene when generation becomes unreliable. We show that a natural candidate, the centered token log-probability increment $\log p(w_t)+H_t$, is the wrong observable for this purpose. Under the model's own sampling law it is a mean-zero martingale by construction, so it measures sampling self-consistency rather than trajectory health and is nearly silent during confident repetition, where both $\log p(w_t)$ and entropy are close to zero. We introduce a training-free decoding controller that combines (i) a degeneration-aware alarm score fusing token uncertainty with explicit verbatim repetition and (ii) a calibrated e-process-inspired sequential detector. The raw product process is Ville-valid under a conditional-mean null, while the deployed CUSUM-floored statistic is treated as an empirical change detector because the score is history-dependent and autocorrelated. On GSM8K with DeepSeek-R1-Distill-Qwen-1.5B in FP16 and INT4, calibration turns a monitor that fires on 93--95% of generations into a selective detector of failing traces ($ϕ\approx 0.3$, precision $\approx 0.6$ against a 0.38 base rate). In this pilot, the controller reduces measured verbatim-degeneration signals and yields a positive but statistically inconclusive INT4 accuracy change from 63% to 69% (paired McNemar $p=0.18$, $n=100$), at a 28% token-budget cost. We also find that non-termination, rather than looping, is the dominant failure mode on GSM8K. The main contribution is methodological: an explanation of why centered token log-probability is inadequate for decoder monitoring and a calibrated, cautiously evaluated replacement.

量化推理解码监控模型可靠性GSM8K

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。