arXiv:2605.26795cs.AI2026-05

固定思维链的词语顺序,局部共现就能带来显著效果。

What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation

论文配图:What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation
图 1 · 摘自论文原文
  • 用局部词序重组实验,发现短距离共现比全局顺序更重要
  • 三词窗口内恢复顺序即可接近完整思维链性能
  • 适合研究模型推理机制或提示工程优化的研究者

思维链提示能提升大语言模型性能,但其作用机制尚不明确。本文从探查时视角出发,固定思维链推理过程,测试哪些文本特征影响最终预测。在多个数据集和模型配置下,随机打乱推理句顺序对准确率影响很小,表明推理步骤的全局顺序并非主要贡献因素。即使打乱所有词序,性能仍远高于无推理基线,说明词汇本身仍具价值。仅恢复短程词序即可显著提升性能,且三词窗口内已获得大部分增益。控制实验证明,答案复制、简单词汇线索、主题上下文或对打乱的鲁棒性均非主要原因。机制分析显示,短窗口增益主要出现在模型早期到中期层,且与答案相关的证据集中于局部文本片段。综合表明,固定推理文本的探查时优势主要来自其包含的词语及短程共现关系。

原文摘要 · Abstract (English)

Chain-of-thought (CoT) prompting enhances large language model performance, yet what drives these gains remains unclear. We study this question from a probe-time perspective: holding CoT rationales fixed, we test which textual properties matter for the final prediction. Across multiple datasets and model configurations, we find that randomizing the order of rationale sentences has little effect on accuracy, suggesting that the global order of reasoning steps is not the main source of the probe-time benefit. Moreover, even when the words in a rationale are randomly reordered, performance remains well above the no-rationale baseline, indicating that the rationale's words remain useful even without their original order. Restoring only short-range word order further improves performance and brings it substantially closer to full CoT. In most settings, much of this local-order gain is already obtained with three-word windows. Control experiments rule out explicit answer copying, simple lexical cues, generic topical context, and general robustness to shuffling as the main explanations. Mechanistic analyses further show that short-window gains are largely formed in early-to-middle model layers, with answer-relevant evidence concentrated in local text spans. Together, these findings support a local co-occurrence activation (LCA) interpretation: the probe-time benefit of fixed rationales arises mainly from the words they contain and short-range word co-occurrences.

思维链提示工程模型机制局部共现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。