通过特征替换验证大模型思维链推理是否真实可信。
How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding
- 用稀疏自编码器提取模型内部单一语义特征，结合激活替换分析推理机制。
- 28亿参数模型中替换思维链特征可显著提升答案置信度（1.2→4.3），7000万参数模型无此效应。
- 揭示了思维链在大模型中引发更模块化、可解释的内部计算结构，适合关注模型可解释性的研究者。
思维链（CoT）提示能提升大语言模型在多步任务中的准确率，但其生成的“思考过程”是否反映真实的内部推理仍不明确。本文首次从特征层面开展因果分析，结合稀疏自编码器与激活替换技术，从Pythia-70M和Pythia-2.8B模型中提取单一语义特征，研究其在解决GSM8K数学问题时的推理行为。将少量CoT推理特征替换至noCoT运行中，可在2.8B模型中显著提高答案对数概率，但在70M模型中无可靠效果，揭示明显的规模阈值。此外，大模型在CoT下激活稀疏性更高，特征可解释性评分也显著上升，例如模型对正确答案的信心从1.2提升至4.3。我们引入patch曲线与随机特征替换基线，表明有用信息不仅存在于前K个高激活特征，而是广泛分布。总体而言，结果表明思维链能在高容量模型中诱导更可解释的内部结构，验证其作为结构化提示方法的有效性。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) prompting boosts Large Language Models accuracy on multi-step tasks, yet whether the generated "thoughts" reflect the true internal reasoning process is unresolved. We present the first feature-level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia-70M and Pythia-2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT-reasoning features into a noCoT run raises answer log-probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear scale threshold. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch-curves and random-feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。