让大模型分步写代码解奥数题，性能远超现有方法
SBSC: Step-By-Step Coding for Improving Mathematical Olympiad Performance
- 大模型分步生成程序求解奥数题，每步利用前序结果
- 在AMC12/AIME/MathOdyssey上分别提升10.7%/8%/12.6%准确率
- 适合需要精确推理的数学竞赛研究与工程应用
我们提出分步编码（SBSC）：一种多轮数学推理框架，使大语言模型（LLMs）能够生成一系列程序来解决奥林匹克级别数学问题。在每一步中，模型利用前序步骤的代码执行结果和程序，生成下一个子任务及对应程序，逐步逼近最终答案。相比现有方法，SBSC提供了更细粒度、灵活且精准的问题求解方式。大量实验表明其在竞赛级数学问题上的有效性。对于Claude-3.5-Sonnet，SBSC（贪婪解码）在AMC12、AIME和MathOdyssey上分别超越现有最先进（SOTA）程序生成推理策略10.7%、8%和12.6%。鉴于SBSC为多轮结构，我们还对比了其贪婪解码与现有SOTA方法自一致性解码的结果，发现其在AMC、AIME和MathOdyssey上分别提升6.2%、6.7%和7.4%。
原文摘要 · Abstract (English)
We propose Step-by-Step Coding (SBSC): a multi-turn math reasoning framework that enables Large Language Models (LLMs) to generate sequence of programs for solving Olympiad level math problems. At each step/turn, by leveraging the code execution outputs and programs of previous steps, the model generates the next sub-task and the corresponding program to solve it. This way, SBSC, sequentially navigates to reach the final answer. SBSC allows more granular, flexible and precise approach to problem-solving compared to existing methods. Extensive experiments highlight the effectiveness of SBSC in tackling competition and Olympiad-level math problems. For Claude-3.5-Sonnet, we observe SBSC (greedy decoding) surpasses existing state-of-the-art (SOTA) program generation based reasoning strategies by absolute 10.7% on AMC12, 8% on AIME and 12.6% on MathOdyssey. Given SBSC is multi-turn in nature, we also benchmark SBSC's greedy decoding against self-consistency decoding results of existing SOTA math reasoning strategies and observe performance gain by absolute 6.2% on AMC, 6.7% on AIME and 7.4% on MathOdyssey.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。