通过整合推理过程，提升大模型在错误修正和预测准确性上的表现。
A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration
- 将推理步骤连贯整合，增强模型纠错能力
- 中间推理步骤出错比最终结果出错更敏感
- 示范中加入正确与错误路径，提升泛化效果
少样本链式思维（CoT）提示已显著提升大语言模型的推理能力。现有理论研究多基于分离式上下文学习（Stepwise ICL），但本文首次从理论上证明：若将前期推理步骤连贯整合（即相干链式思维，Coherent CoT），Transformer 模型能获得更强的错误修正能力和更准确的预测。由于连贯推理改变了模型行为，我们进一步分析了在推理阶段示范样本被污染时模型的敏感性。理论结果表明，模型对中间推理步骤的错误比对最终结果的错误更为敏感。基于此，我们提出在示范中同时引入正确与错误推理路径的方法。实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Few-shot Chain-of-Thought (CoT) prompting has demonstrated strong performance in improving the reasoning capabilities of large language models (LLMs). While theoretical investigations have been conducted to understand CoT, the underlying transformer used in these studies isolates the CoT reasoning process into separated in-context learning steps (Stepwise ICL). In this work, we theoretically show that, compared to Stepwise ICL, the transformer gains better error correction ability and more accurate predictions if the reasoning from earlier steps (Coherent CoT) is integrated. Given that this coherent reasoning changes the behavior of the transformer, we further investigate the sensitivity of the transformer with Coherent CoT when the demonstration examples are corrupted at the inference stage. Our theoretical results indicate that the transformer is more sensitive to errors in intermediate reasoning steps than the final outcome. Building upon this observation, we propose an improvement on CoT by incorporating both correct and incorrect reasoning paths in the demonstration. Our experiments validate the effectiveness of the proposed approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。