用人工标注的解题流程数据，让大模型学会像人一样思考。
REL: Working out is all you need
- 构建专门的解题过程数据集，引导模型模仿人类推理步骤。
- 在数学和逻辑任务上，模型解题准确率提升显著，接近O1水平。
- 适合想提升模型推理能力的研究者或开发者使用。
近期发展，特别是OpenAI的O1模型，展示了大型语言模型（LLMs）在复杂推理任务中的巨大潜力。通过分析O1的输出及提供的思维链（CoT）示例，我们观察到其解决问题的方式具有明显的人类特征：系统性地头脑风暴、测试假设、验证结果并规划全面解决方案。这种高级推理能力在其他先进语言模型中仍不明显。本文假设这一性能差距源于当前训练数据中高质量推理过程的稀缺。我们证明，通过构建专注于明确问题解决流程（“已解决的方案”）的专用数据集，可显著激发现有模型的规划能力。此外，我们提出推理增强循环（REL），一种生成合成解决方案的方法。
原文摘要 · Abstract (English)
Recent developments, particularly OpenAI's O1 model, have demonstrated the remarkable potential of Large Language Models (LLMs) for complex reasoning tasks. Through analysis of O1's outputs and provided sample Chain-of-Thought (CoT) demonstrations, we observe that it approaches problem-solving in a distinctly human-like manner, systematically brainstorming ideas, testing hypotheses, verifying results, and planning comprehensive solutions. These sophisticated reasoning capabilities remain notably absent in other state-of-the-art language models. In this paper, we hypothesize that this performance gap stems from the limited availability of high-quality reasoning process data in current training sets. We demonstrate that by constructing a specialized dataset focused on explicit problem-solving workflows ("worked solutions"), we can elicit substantially improved planning capabilities from existing models. Additionally, we propose the Reasoning Enhancement Loop (REL), a method for generating synthetic worked solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。