开源数学推理模型MiroMind-M1,用多阶段优化提升推理能力与效率。
MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimization
- 分两阶段训练:71.9万题SFT+6.2万题强化学习,增强逻辑链生成
- 在AIME24/25和MATH基准上性能达开源模型顶尖水平,32B版更高效
- 完整开源训练数据、模型与配置,支持复现与社区发展
大语言模型已从文本生成进化到跨领域高级推理,其中数学推理作为典型代表,需精准多步逻辑与抽象思维,可泛化至其他任务。尽管闭源推理模型如GPT-o3表现优异,但其专有性限制了透明度与可复现性。虽诸多开源项目试图填补空白,但多数缺少关键数据集与训练配置,阻碍复现。为此,我们推出MiroMind-M1系列,基于Qwen-2.5的全开源推理模型,在性能上达到或超越现有开源模型。模型经两阶段训练:首先在71.9万道经验证的数学推理题(含正确思维链)上进行SFT;随后在6.2万道高难度且可验证题目上进行强化学习(RLVR)。为提升训练鲁棒性与效率,提出上下文感知的多阶段策略优化算法,融合长度渐进训练与自适应重复惩罚,促进上下文敏感的强化学习。模型在AIME24、AIME25与MATH基准上达到当前最优或具有竞争力的表现,并在7B与32B规模下展现更优的令牌效率。为推动可复现性,我们公开完整资源:模型(MiroMind-M1-SFT-7B, MiroMind-M1-RL-7B, MiroMind-M1-RL-32B)、数据集(MiroMind-M1-SFT-719K, MiroMind-M1-RL-62K)及所有训练与评估配置。期望这些资源助力后续研究并促进社区进步。
原文摘要 · Abstract (English)
Large language models have recently evolved from fluent text generation to advanced reasoning across diverse domains, giving rise to reasoning language models. Among these domains, mathematical reasoning serves as a representative benchmark as it requires precise multi-step logic and abstract reasoning, which can be generalized to other tasks. While closed-source RLMs such as GPT-o3 demonstrate impressive reasoning capabilities, their proprietary nature limits transparency and reproducibility. Although many open-source projects aim to close this gap, most of them lack sufficient openness by omitting critical resources such as datasets and detailed training configurations, which hinders reproducibility. To contribute toward greater transparency in RLM development, we introduce the MiroMind-M1 series, a set of fully open-source RLMs built on the Qwen-2.5 backbone that match or exceed the performance of existing open-source RLMs. Specifically, our models are trained in two stages: SFT on a carefully curated corpus of 719K math-reasoning problems with verified CoT trajectories, followed by RLVR on 62K challenging and verifiable problems. To enhance the robustness and efficiency of the RLVR process, we introduce Context-Aware Multi-Stage Policy Optimization, an algorithm that integrates length-progressive training with an adaptive repetition penalty to encourage context-aware RL training. Our model achieves state-of-the-art or competitive performance and superior token efficiency among Qwen-2.5-based open-source 7B and 32B models on the AIME24, AIME25, and MATH benchmarks. To facilitate reproducibility, we release the complete stack: models (MiroMind-M1-SFT-7B, MiroMind-M1-RL-7B, MiroMind-M1-RL-32B); datasets (MiroMind-M1-SFT-719K, MiroMind-M1-RL-62K); and all training and evaluation configurations. We hope these resources will support further research and foster community advancement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。