通过分层思维模板提升大模型数学推理能力，效果超越多个顶级模型。
ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
- 构建通用思维模板库，分层优化推理路径规划。
- 在MATH基准上达91.2%准确率，AIME上解决率56.7%，显著领先。
- 适合需要可解释推理的高阶数学问题求解场景。
我们提出通过扩展思维模板实现分层大模型推理，能有效优化推理搜索空间，并超越OpenAI o1-preview和DeepSeek V3等强大模型的数学推理能力。仅用8张GPU训练的ReasonFlux-32B引入三项创新：(i) 包含约500个高层级思维模板的结构化通用模板库，可泛化至相似或相关推理任务；(ii) 在思维模板序列上进行分层强化学习，而非长链条自洽推理（CoT），优化基础模型以规划最优模板轨迹，逐步处理复杂问题；(iii) 一种全新的推理缩放系统，可在推理时自适应扩展思维模板。其模板轨迹具有比DeepSeek-R1和o3-mini更强的可解释性结构。在MATH基准上，准确率达到91.2%，超过o1-preview 6.7%；在美国数学奥林匹克（AIME）基准上，平均解决率达56.7%，分别超越o1-preview和DeepSeek-V3 27%和45%。代码已开源。
原文摘要 · Abstract (English)
We present that hierarchical LLM reasoning via scaling thought templates can effectively optimize the reasoning search space and outperform the mathematical reasoning capabilities of powerful LLMs like OpenAI o1-preview and DeepSeek V3. We train our ReasonFlux-32B model with only 8 GPUs and introduces three innovations: (i) a structured and generic thought template library, containing around 500 high-level thought templates capable of generalizing to similar or relevant reasoning problems; (ii) performing hierarchical reinforcement learning on a sequence of thought templates instead of long CoTs, optimizing a base LLM to plan out an optimal template trajectory for gradually handling complex problems; (iii) a brand new inference scaling system that enables hierarchical LLM reasoning by adaptively scaling thought templates at inference time. With a template trajectory containing more explainable reasoning structures than DeepSeek-R1 and o3-mini, our ReasonFlux-32B significantly advances math reasoning capabilities to state-of-the-art levels. Notably, on the MATH benchmark, it achieves an accuracy of 91.2% and surpasses o1-preview by 6.7%. On the USA Math Olympiad (AIME) benchmark, ReasonFlux-32B solves an average of 56.7% of problems, surpassing o1-preview and DeepSeek-V3 by 27% and 45%, respectively. Code: https://github.com/Gen-Verse/ReasonFlux
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。