用可读的代码形式让思维链更透明，便于分析错误和模块性。
Watch Your Steps: Observable and Modular Chains of Thought
- 将思维链转为带命名和输入输出的Python格式步骤
- 在BIG-Bench Hard上23个任务均表现强劲
- 能识别非局部错误并验证步骤的模块性，适合可解释性研究
我们提出一种名为程序追踪提示（Program Trace Prompting）的思维链（CoT）提示变体，使解释过程更可观测，同时保持原思维链的强大、通用与灵活。该方法将少量示例的思维链演示封装为基于Python的正式语法：每一步都命名，定义输入输出行为，并用形式化步骤替换原示例中的自然语言解释。该方法适用于多种任务，在包含23个多样化任务的BIG-Bench Hard基准上取得优异表现。更重要的是，通过这种结构化方式，我们首次识别出‘非局部错误’（即错误地学习示范中的推理模式），并提出验证思维链步骤模块性的方法，推动对模型推理过程的深入分析。
原文摘要 · Abstract (English)
We propose a variant of chain of thought (CoT) prompting called Program Trace Prompting that makes explanations more observable while preserving the power, generality and flexibility of CoT. In our approach, few-shot CoT demonstrations are wrapped in a formal syntax based on Python, and each prompt: identifies and names steps; defines the input/output behavior of steps; and replaces CoT explanations of in-context examples with chains of these formalized steps on the same examples. Program Trace Prompting is applicable to many tasks, achieving strong results on the 23 diverse tasks in the BIG-Bench Hard benchmark. More importantly, by instrumenting explanations in this way, we enable new types of analysis. In particular, we identify "non-local errors" (which correspond to incorrectly learning the reasoning method illustrated in the demonstrations) as an unaddressed issue in CoT learning, and we present methods for verifying the modularity of steps in a CoT explanation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。