arXiv:2608.28859cs.LGcs.AI2026-08中稿 · COLM

通过内化因果干预,让模型在知道答案后及时停止思考。

The Halt Vector: Internalizing a Causal Steering Intervention for Efficient Reasoning

  • 在第18层引入停顿向量控制思考长度,优于全局长度惩罚。
  • 无需强化学习,仅用24个问题训练,即在5个新基准上减少约25%的冗余推理。
  • 解决难题时模型不停止的病理问题,适合优化推理效率的研究者。

推理模型不会在知道答案后立即停止。以 DeepSeek-R1-Distill-Qwen-7B 为例,思维链长度约为模型自身答案概率稳定时间的两倍,且冗余部分可削减量因问题而异,因此全局长度惩罚无法有效消除。本文将因果可解释性发现内化至模型权重中,提出一种‘停顿向量’机制:该向量为第18层的一个均值差方向,其控制强度决定模型思考时长,而复制值轴则无效。直接嵌入该干预存在挑战:沿该方向投影会破坏下游读取器依赖的非轴维度,反而延长生成。正确做法是固定这些维度为自然值,重建整个受控激活。仅用24个问题训练,未使用强化学习,该方法在五个未见基准上保持精度的同时,平均减少约四分之一的推理步骤,且剪枝程度与各题可削减空间高度一致(相关系数0.70)。此外,它还缓解了随难度增长的非终止病态,该病态会因解码时置信度钩子而加剧。本文不宣称在原始权衡上超越调优后的长度惩罚或解码时早停,核心贡献在于如何获得该停顿机制。

原文摘要 · Abstract (English)

Reasoning models do not stop when they know the answer. On DeepSeek-R1-Distill-Qwen-7B the chain of thought runs about twice as long as the model's own answer probability takes to settle, and how much of that excess is removable varies from problem to problem, so a global length penalty cannot take it out. We take it out by internalizing a causal interpretability finding into the weights. The mechanism is a halt vector: a difference-of-means direction at layer 18 of this model whose steering strength controls how long it thinks, while a replicated value axis does nothing. Installing that intervention in the weights is harder than it looks. Maximizing the scalar projection onto the direction corrupts the off-axis dimensions a frozen downstream reader depends on, and generation gets longer instead of shorter; what works is reconstructing the whole steered activation with those dimensions pinned to their natural values. Fit from 24 problems and no reinforcement learning, the halt removes about a quarter of the thinking at held accuracy across five unseen benchmarks, and the cut tracks each problem's own removable slack at 0.70. It also closes a non-termination pathology that grows with difficulty and that a decoding-time confidence hook makes worse. We do not claim to beat a well-tuned length penalty or decoding-time early exit on the raw trade-off; the contribution is how the halt is obtained.

推理优化因果干预停顿向量模型效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。