14B参数模型在单张消费级显卡上高效运行,专为中文K-12数学教育优化。
Confucius3-Math: A Lightweight High-Performance Reasoning LLM for Chinese K-12 Mathematics Learning
- 基于强化学习后训练,针对中文数学教育任务优化
- 在多项数学推理任务中表现超越更大模型,实现性能领先
- 适合教育机构与教师用于低成本智能辅导系统搭建
我们提出Confucius3-Math,一个拥有140亿参数的开源大语言模型,可在单张消费级GPU上高效运行,并在多个数学推理任务中达到最先进水平,性能优于许多参数量显著更大的模型。作为推动AI赋能教育与知识传播使命的一部分,该模型专注于中国中小学数学学习,通过大规模强化学习后训练构建,对齐国家课程标准,能够以低成本高效解决主流中文K-12数学问题。本文分享了模型开发过程中的技术方案、挑战与应对策略,提出三项核心技术创新:目标熵正则化、近期样本恢复和策略特定难度加权。这些方法包含新型熵正则化机制、新颖的数据调度策略以及改进的组相对优势估计器,共同显著提升了强化学习训练的稳定性、数据利用效率并增强性能。本工作证明了在特定领域以低资源成本构建强推理模型的可行性。模型与代码已开源,详见https://github.com/netease-youdao/Confucius3-Math。
原文摘要 · Abstract (English)
We introduce Confucius3-Math, an open-source large language model with 14B parameters that (1) runs efficiently on a single consumer-grade GPU; (2) achieves SOTA performances on a range of mathematical reasoning tasks, outperforming many models with significantly larger sizes. In particular, as part of our mission to enhancing education and knowledge dissemination with AI, Confucius3-Math is specifically committed to mathematics learning for Chinese K-12 students and educators. Built via post-training with large-scale reinforcement learning (RL), Confucius3-Math aligns with national curriculum and excels at solving main-stream Chinese K-12 mathematical problems with low cost. In this report we share our development recipe, the challenges we encounter and the techniques we develop to overcome them. In particular, we introduce three technical innovations: Targeted Entropy Regularization, Recent Sample Recovery and Policy-Specific Hardness Weighting. These innovations encompass a new entropy regularization, a novel data scheduling policy, and an improved group-relative advantage estimator. Collectively, they significantly stabilize the RL training, improve data efficiency, and boost performance. Our work demonstrates the feasibility of building strong reasoning models in a particular domain at low cost. We open-source our model and code at https://github.com/netease-youdao/Confucius3-Math.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。