70亿参数模型通过预训练与后训练优化，实现超越32B大模型的推理能力。
MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining
- 分三阶段混合数据预训练，加入多标记预测提升推理潜力。
- 13万道可验证数学编程题用于强化学习，解决奖励稀疏问题。
- 仅70亿参数却超越32B模型，适合需要高效推理的场景。
我们提出MiMo-7B，一个专为推理任务设计的大语言模型，通过预训练与后训练双重优化。预训练阶段，采用三阶段数据混合策略并改进数据预处理，基于25万亿标记训练基础模型，引入多标记预测目标以提升性能和加速推理。后训练阶段，构建包含13万道可验证数学与编程问题的数据集，结合测试难度驱动的代码奖励机制缓解稀疏奖励问题，并通过策略性数据重采样稳定训练。大量评估显示，MiMo-7B-Base具备卓越推理潜力，表现优于更大规模的32B模型。最终的强化学习调优模型MiMo-7B-RL在数学、编程与通用推理任务上均超越OpenAI o1-mini。模型权重已开源：https://github.com/xiaomimimo/MiMo。
原文摘要 · Abstract (English)
We present MiMo-7B, a large language model born for reasoning tasks, with optimization across both pre-training and post-training stages. During pre-training, we enhance the data preprocessing pipeline and employ a three-stage data mixing strategy to strengthen the base model's reasoning potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional Multi-Token Prediction objective for enhanced performance and accelerated inference speed. During post-training, we curate a dataset of 130K verifiable mathematics and programming problems for reinforcement learning, integrating a test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and employing strategic data resampling to stabilize training. Extensive evaluations show that MiMo-7B-Base possesses exceptional reasoning potential, outperforming even much larger 32B models. The final RL-tuned model, MiMo-7B-RL, achieves superior performance on mathematics, code and general reasoning tasks, surpassing the performance of OpenAI o1-mini. The model checkpoints are available at https://github.com/xiaomimimo/MiMo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。