arXiv:2607.20129cs.AI2026-07被引 1

量化小模型推理时用监控重解,提升错误修正能力。

CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning

  • 通过窗口化不确定性和退化特征,动态判断是否需回滚重解。
  • 在240对测试中准确率从82%提升至88%,13次纠错7次误纠。
  • 适合关注推理质量优化的模型部署与调试人员使用。

量化的小型自回归推理模型可能陷入长周期重复或无效轨迹,但推理时计算资源通常未根据轨迹发展情况分配。基于早期的逐标记e-CUSUM控制器,本文提出MGT-B(监控引导的测试时回溯),一种外部控制器,将重叠窗口内的预采样不确定性与退化特征映射为位置条件下的经验尾部概率,采用CUSUM形状的重置累积混合投注因子,并在触发警报后估计回滚点,恢复标记与键值缓存状态并执行受限重解。为检验效果是否在手动选择日志阈值h=10之后仍持续存在,我们回溯性排除260个阈值前出现的问题标识符,保留每个剩余标识符的首个阈值后配对,形成240对的时间审计集。在此集上,准确率由82/240提升至88/240(+2.50个百分点;13次纠正,7次退步;精确McNemar p = 0.2632;配对自助法95%置信区间[-1.25, +6.25])。更广泛的467对历史覆盖集(种子匹配)准确率由146/467升至167/467(+4.50点;McNemar p = 0.000753),但包含200个阈值前或阈值期间可用的种子-1标识符,仅作为探索性估计。所有316个无警报输出与原始版本一致,而151个触发警报轨迹中含29次纠正和8次退步。两项分析均非确认性结果,且经验因子未被确立为有效e过程或e检测器。结果支持在所研究的MATH-500设置下采用选择性监控与修复机制,而非普遍或理论认证的推理改进。

原文摘要 · Abstract (English)

Quantized small autoregressive reasoning models can enter long, repetitive, or unproductive trajectories, yet inference-time compute is usually allocated without observing how a trajectory develops. Building on an earlier token-level e-CUSUM controller, we develop MGT-B (Monitoring-Guided Test-time Backtracking), a revised external controller that maps overlapping windows of pre-sampling uncertainty and degeneration features to position-conditional empirical tail probabilities, accumulates mixture betting factors with a CUSUM-shaped reset, and responds to an alarm by estimating a rollback point, restoring token and key-value-cache state, and performing constrained re-decoding. To audit whether the effect persists on problem identities first observed after the manual choice of log threshold h = 10, we retrospectively exclude 260 IDs present in pre-threshold artifacts and retain the chronologically first post-threshold pair for each remaining ID, yielding a 240-pair chronology-audit set. On this set, accuracy changes from 82/240 to 88/240 (+2.50 percentage points; 13 corrections, 7 regressions; exact McNemar p = 0.2632; paired bootstrap 95% interval [-1.25, +6.25]). A broader 467-pair historical-coverage set of seed-matched pairs changes accuracy from 146/467 to 167/467 (+4.50 points; McNemar p = 0.000753), but includes 200 seed-1 IDs available before or during threshold selection and is reported only as an exploratory estimate. All 316 no-alarm outputs in the 467-pair set are identical to vanilla, while the 151 alarmed trajectories contain 29 corrections and 8 regressions. Neither analysis is confirmatory, and the empirical factors are not established as a valid e-process or e-detector. The results support a selective monitoring-and-repair mechanism for the studied MATH-500 setting, rather than a general or theoretically certified reasoning improvement.

量化推理优化监控重解小模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。