用执行轨迹验证代码推理，让大模型学会真正正确的思考过程。
Generating Verifiable Chain of Thoughts from Exection-Traces
- 通过记录代码执行轨迹，生成可验证的自然语言推理链。
- 在三个基准上提升超20分，验证数据质量直接影响模型表现。
- 适合需要可靠代码推理的开发者和研究者使用。
让语言模型正确理解代码推理，需依赖可验证的训练数据。当前合成的思维链（CoT）数据多由教师模型生成，看似合理但不可验证，导致模型习得逻辑错误的推理模式。为此，我们构建了一套流水线：通过代码插桩捕获执行轨迹，将轨迹转化为自然语言叙述，并与原始轨迹交叉核验。系统化生成了54,000条双向验证的推理链，涵盖从输入到输出及反向推理。在该数据上微调的模型在LiveCodeBench-Exec上最高提升+26.6，CruxEval上+22.2，HumanEval上+19.5，证明验证质量直接决定推理与代码生成能力。完整合成流程已开源：https://github.com/IBM/verified-code-cot/
原文摘要 · Abstract (English)
Getting language models to reason correctly about code requires training on data where each reasoning step can be checked. Current synthetic Chain-of-Thought (CoT) training data often consists of plausible-sounding explanations generated by teacher models, and not verifiable accounts of actual program behavior. Models trained on such data learn logically flawed reasoning patterns despite syntactic correctness. To address this, we build a pipeline that generates execution-trace-verified CoT rationales by instrumenting code to capture traces, narrating them into natural language, and cross-checking each narration against the original trace. We systematically create 54,000 verified, bi-directional rationales that teach models to reason both forward (input$\rightarrow$output) and backward (output$\rightarrow$input). Models fine-tuned on our verified data achieve substantial improvements, with a peak gain of +26.6 on LiveCodeBench-Exec, +22.2 on CruxEval, and +19.5 on HumanEval across our fine-tuned models, demonstrating that verification quality directly determines both reasoning and code generation capabilities. Complete synthesis pipeline is avilable as open-source: https://github.com/IBM/verified-code-cot/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。