让大模型通过算法题训练推理能力，不依赖代码实现。
Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code
- 用精心筛选的数据和强化学习引导模型发现最优推理路径。
- 在17个基准上显著提升，最大提升达35.9%。
- 适合想提升模型逻辑推理能力的研究者与开发者。
提升大模型的推理能力仍是研究重点。一种有效方法是要求模型通过逐步模拟代码执行来推导输出，但直接使用代码会导致模型过度依赖复杂数据结构与算法，即使在简单任务中也容易过拟合于算法模式而非核心推理逻辑。为此，我们提出TeaR，旨在更有效地训练大模型的推理能力。TeaR结合精细的数据筛选与强化学习，引导模型在代码相关任务中探索最优推理路径，从而增强通用推理能力。我们在两种基础模型和三种长思维链蒸馏模型上进行广泛实验，模型参数量从15亿到320亿不等，涵盖数学、知识、代码与逻辑推理共17个基准。结果一致显示性能显著提升：Qwen2.5-7B模型提升35.9%，R1-Distilled-7B提升5.9%。
原文摘要 · Abstract (English)
Enhancing reasoning capabilities remains a central focus in the LLM reasearch community. A promising direction involves requiring models to simulate code execution step-by-step to derive outputs for given inputs. However, as code is often designed for large-scale systems, direct application leads to over-reliance on complex data structures and algorithms, even for simple cases, resulting in overfitting to algorithmic patterns rather than core reasoning structures. To address this, we propose TeaR, which aims at teaching LLMs to reason better. TeaR leverages careful data curation and reinforcement learning to guide models in discovering optimal reasoning paths through code-related tasks, thereby improving general reasoning abilities. We conduct extensive experiments using two base models and three long-CoT distillation models, with model sizes ranging from 1.5 billion to 32 billion parameters, and across 17 benchmarks spanning Math, Knowledge, Code, and Logical Reasoning. The results consistently show significant performance improvements. Notably, TeaR achieves a 35.9% improvement on Qwen2.5-7B and 5.9% on R1-Distilled-7B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。