用强化学习提升大模型推理能力，32B版准确率提至72.8%
Skywork Open Reasoner 1 Technical Report
- 基于深度思维链的强化学习训练，优化长链条推理
- 32B模型在AIME24/25上准确率从57.8%升至72.8%（+15.0%）
- 开源全部代码数据，适合研究推理增强与模型训练
DeepSeek-R1的成功凸显了强化学习（RL）在提升大语言模型（LLM）推理能力中的关键作用。本文提出Skywork-OR1，一种高效且可扩展的长链思维（CoT）模型强化学习实现方案。基于DeepSeek-R1-Distill模型系列，我们的RL方法显著提升性能：32B模型在AIME24、AIME25和LiveCodeBench上的平均准确率从57.8%提升至72.8%（+15.0%），7B模型则从43.6%提升至57.5%（+13.9%）。Skywork-OR1-32B在AIME24和AIME25上超越DeepSeek-R1与Qwen3-32B，LiveCodeBench表现相当。Skywork-OR1-7B与Skywork-OR1-Math-7B在同规模模型中展现出竞争力。我们通过全面消融实验验证训练流程核心组件有效性，并深入研究熵坍缩现象，识别影响熵动态的关键因素，证明缓解过早熵坍缩对提升测试性能至关重要。为支持社区研究，我们完全开源模型权重、训练代码与训练数据集。
原文摘要 · Abstract (English)
The success of DeepSeek-R1 underscores the significant role of reinforcement learning (RL) in enhancing the reasoning capabilities of large language models (LLMs). In this work, we present Skywork-OR1, an effective and scalable RL implementation for long Chain-of-Thought (CoT) models. Building on the DeepSeek-R1-Distill model series, our RL approach achieves notable performance gains, increasing average accuracy across AIME24, AIME25, and LiveCodeBench from 57.8% to 72.8% (+15.0%) for the 32B model and from 43.6% to 57.5% (+13.9%) for the 7B model. Our Skywork-OR1-32B model surpasses both DeepSeek-R1 and Qwen3-32B on the AIME24 and AIME25 benchmarks, while achieving comparable results on LiveCodeBench. The Skywork-OR1-7B and Skywork-OR1-Math-7B models demonstrate competitive reasoning capabilities among models of similar size. We perform comprehensive ablation studies on the core components of our training pipeline to validate their effectiveness. Additionally, we thoroughly investigate the phenomenon of entropy collapse, identify key factors affecting entropy dynamics, and demonstrate that mitigating premature entropy collapse is critical for improved test performance. To support community research, we fully open-source our model weights, training code, and training datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。