将专家棋手的思考过程蒸馏给小模型,让其学会解释性下棋。
Grounded Chess Reasoning in Language Models via Master Distillation
- 通过蒸馏专家系统完整推理链,生成可解释的逐步推理解释。
- 40亿参数模型在棋局预测上达到48.1%准确率,超越多数开源与主流闭源模型。
- 生成答案所需词元数比基线少两个数量级,适合资源受限场景。
语言模型在训练数据稀少但专用系统表现优异的特定领域中,常缺乏具象化推理能力。我们提出一种通用框架,将专家系统推理过程蒸馏为自然语言的思维链解释,使小型模型能够获得领域专长及生成忠实、具象化解释的能力。不同于仅蒸馏最终输出,本方法捕获完整的推理过程,将专家系统的黑箱计算转化为透明的分步推理解释。我们在国际象棋这一经典推理领域验证该方法,尽管语言模型在此领域仍表现不佳。我们的40亿参数模型C1从接近零的基线提升至48.1%准确率,超越所有开源模型和多数前沿闭源系统。值得注意的是,C1不仅超过其蒸馏教师模型,且生成解题所需的词元数比基线少两个数量级。与以往仅预测最优走法的神经象棋方法不同,C1能生成揭示战略思路的可解释解法。我们的流程结合监督微调与强化学习,并采用主题均衡的数据采样策略以实现全面战术覆盖。主蒸馏方法展示了如何将专家级知识注入小型模型,为语言模型在低优化领域(RLVR)中释放潜力提供了可复用范式。
原文摘要 · Abstract (English)
Language models often lack grounded reasoning capabilities in specialized domains where training data is scarce but bespoke systems excel. We introduce a general framework for distilling expert system reasoning into natural language chain-of-thought explanations, enabling compact models to acquire domain expertise and the ability to generate faithful, grounded explanations. Rather than distilling only final outputs, we capture the full reasoning process, transforming opaque expert computations into transparent, step-by-step explanations. We demonstrate this approach in chess, a canonical reasoning domain where language models continue to underperform. Our 4B parameter model, C1, advances from a near-zero baseline to 48.1\% accuracy, outperforming all open-source models and most frontier proprietary systems. Notably, C1 surpasses its distillation teacher and generates solutions in two orders of magnitude fewer tokens than baselines. Unlike prior neural chess approaches that predict only best moves, C1 generates explainable solutions revealing strategic reasoning. Our pipeline combines supervised fine-tuning and reinforcement learning with theme-balanced data sampling for comprehensive tactical coverage. Master Distillation demonstrates how to inject expert-level knowledge into compact models for under-optimized domains, offering a recipe for unlocking RLVR where LLMs lack sufficient base capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。