arXiv:2607.09786cs.AIcs.CL2026-07

压缩思维链虽降成本,却让模型更难被监控。

Length Penalties Make Chain-of-Thought Less Monitorable

  • 用长度惩罚压缩思维链以降低推理开销。
  • 压缩后模型对提示的依赖仍近基线水平,监控率下降20%以上。
  • 适合关注模型可解释性与安全性的研究者。

为减少过度推理和降低推理成本,研究者现对推理模型施加思维链长度惩罚。我们发现,此类惩罚会损害模型可监控性。较短的思维链提及误导性提示的频率更低,但提示仍显著影响模型答案。我们在保持目标思维链长度的前提下,训练Qwen3 4B和Qwen3 14B模型,并在独立的MMLU Pro R数据集及四个迁移基准上评估其表现。压缩显著减少推理标记数并保留大部分多选准确率,但提示影响程度接近基线。在最短目标链长下,Qwen3 14B和Qwen3 4B的最低忠实度分别降至基线的63.1%和69.4%。监控器原始提示检测率从69%降至49%,从60%降至48%。通过随机删除未压缩基线链中句子使长度匹配,结果表明:在两种模型规模及全部五个评估分布中,压缩链提及提示的频率比长度匹配基线低7至35个百分点。因此,我们识别出压缩与可监控性之间的权衡边界——减少推理成本所损失的证据,远超仅由链长缩短带来的预期。

原文摘要 · Abstract (English)

To curb overthinking and reduce inference costs, researchers now train reasoning models with penalties on chain of thought length. We find that these penalties degrade monitorability. Shorter chains of thought mention misleading hints less often, but the hints still influence the models' answers. We train Qwen3 4B and Qwen3 14B to produce different target chain lengths, then evaluate them using biasing hint interventions on held out MMLU Pro R data and four transfer benchmarks. Compression reduces reasoning tokens and preserves most multiple choice accuracy, while hint influence remains near baseline. At the shortest target chain length, lower bound faithfulness drops to 63.1 percent of baseline for Qwen3 14B and 69.4 percent for Qwen3 4B. The monitor's raw hint detection rate falls from 69 percent to 49 percent and from 60 percent to 48 percent, respectively. To separate length from content, we randomly delete sentences from uncompressed baseline chains until the remaining text matches the compressed length. Across both Qwen3 model sizes and all five evaluation distributions, compressed chains still mention the hint 7 to 35 percentage points less often than these length matched baselines. We therefore identify a compression and monitorability frontier where reducing reasoning costs removes more evidence than shorter traces alone would predict.

思维链模型监控推理效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。