揭秘思维链如何通过模板引导模型推理，提升表现。
How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation
- 逆向追踪解码、投影和激活阶段的信息流，揭示机制。
- 高模板遵循度显著提升性能，思维链像解码空间剪枝器。
- 任务类型决定神经元活跃度：开放域降活，封闭域增活。
思维链（CoT）提示显著增强模型推理能力，但其内部机制仍不清晰。我们通过逆向追踪解码、投影和激活阶段的信息流，分析了CoT的运行原理。定量分析表明，CoT可能作为解码空间的剪枝器，利用答案模板引导输出生成，且模板遵循度越高，性能越优。此外，我们意外发现CoT对神经元激活具有任务依赖性：在开放域任务中降低神经元激活，在封闭域任务中则增加。这些发现为理解机制提供了新框架，并为设计更高效、鲁棒的提示提供关键洞见。代码与数据已公开于https://anonymous.4open.science/r/cot-D247。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting significantly enhances model reasoning, yet its internal mechanisms remain poorly understood. We analyze CoT's operational principles by reversely tracing information flow across decoding, projection, and activation phases. Our quantitative analysis suggests that CoT may serve as a decoding space pruner, leveraging answer templates to guide output generation, with higher template adherence strongly correlating with improved performance. Furthermore, we surprisingly find that CoT modulates neuron engagement in a task-dependent manner: reducing neuron activation in open-domain tasks, yet increasing it in closed-domain scenarios. These findings offer a novel mechanistic interpretability framework and critical insights for enabling targeted CoT interventions to design more efficient and robust prompts. We released our code and data at https://anonymous.4open.science/r/cot-D247.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。