通过分析模型内部思考流,精准识别推理关键步骤。
Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression

- 基于模型内部残差流波动,量化每个词对答案的贡献度。
- 在四个基准和四类模型上均优于现有方法,压缩后准确率更高。
- 适合需要高效推理的场景,如资源受限部署或实时问答系统。
链式思维(CoT)推理提升多步问题求解能力,但长推理序列会增加计算开销。基于词元的CoT压缩通过剪枝推理链来降低开销,而词元选择成为核心挑战。现有方法多依赖外部评分器或间接启发信号,与模型内部答案计算关联较弱。本文提出一种模型内部视角:当模型生成答案时,每个推理词元会在残差流(即模型的“思维流”)中留下波动,其幅度反映该词元对答案计算的贡献。基于此,我们提出 extsc{MIST}(Model-Internal Saliency for Token-level CoT compression),从两个互补维度定义词元重要性:必要性(移除该词元后答案概率下降程度)与充分性(仅保留该贡献时答案概率提升程度),结合两者得到统一重要性得分用于剪枝。在四个推理基准和四种模型上, extsc{MIST} 均稳定优于基线方法,表明模型内部显著性可有效代理推理词元的重要性。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) reasoning improves multi-step problem solving, but long reasoning traces inflate inference cost. Token-level CoT compression reduces this cost by pruning full reasoning chains into shorter traces for model adaptation, making token selection the central challenge. Existing methods often rely on external scorers or heuristic signals only indirectly tied to the model's internal answer computation. We instead adopt a model-internal perspective: as the model forms an answer, each reasoning token leaves a ripple in the residual stream, the model's \emph{stream of thought}, and the magnitude of this ripple reflects the token's contribution to the answer computation. Building on this view, we propose \textsc{MIST} (Model-Internal Saliency for Token-level CoT compression), which defines token importance along two complementary axes: \emph{necessity}, the drop in answer likelihood when a token's internal contribution is removed, and \emph{sufficiency}, the gain in answer likelihood when that contribution alone is provided. Combining the two yields a unified importance score for pruning. Across four reasoning benchmarks and four models, \textsc{MIST} consistently outperforms baseline methods, suggesting that model-internal saliency provides an effective proxy for reasoning-token importance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。