用自监督方法提升低资源语言数学推理能力
Crosslingual On-Policy Self-Distillation for Multilingual Reasoning
- 同一模型作师生，通过跨语言上下文迁移高资源语言推理能力
- 在17种非洲低资源语言上显著提升数学推理准确率
- 特别适合多语言推理与低资源场景下的模型优化
大型语言模型在数学推理上取得显著进展，但不同语言间表现差异明显，低资源语言性能远低于高资源语言。为此，我们提出跨语言在线策略自蒸馏（COPSD），将模型自身在高资源语言中的推理行为迁移到低资源语言。该方法使用同一模型作为学生和教师：学生仅看到低资源语言的问题，而教师则接收包含问题翻译和英文参考解答的跨语言上下文。训练时最小化学生自身采样轨迹上的全分布词级差异，提供密集监督，避免仅基于结果的强化学习带来的稀疏性和不稳定性。在17种低资源非洲语言上的实验表明，COPSD在不同模型规模下均能持续提升数学推理性能，显著优于群组相对策略优化（GRPO）。进一步分析显示，该方法提升了答案格式一致性，增强了测试时缩放能力，并在更难的多语言推理基准上实现良好泛化，尤其对低资源语言带来显著提升。代码与数据已公开于：https://github.com/cisnlp/COPSD。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved remarkable progress in mathematical reasoning, but this ability is not equally accessible across languages. Especially low-resource languages exhibit much lower reasoning performance. To address this, we propose Crosslingual On-Policy Self-Distillation (COPSD), which transfers a model's own high-resource reasoning behavior to low-resource languages. COPSD uses the same model as student and teacher: the student sees only the low-resource problem, while the teacher receives privileged crosslingual context, including the problem translation and reference solution in English. Training minimizes full-distribution token-level divergence on the student's own rollouts, providing dense supervision while avoiding the sparsity and instability of outcome-only reinforcement learning (RL). Experiments on 17 low-resource African languages show that COPSD consistently improves low-resource mathematical reasoning across model sizes and substantially outperforms Group Relative Policy Optimization (GRPO). Further analyses show that COPSD improves answer-format adherence, strengthens test-time scaling, and generalizes to harder multilingual reasoning benchmarks, with especially large gains for lower-resource languages. We make our code and data available at: https://github.com/cisnlp/COPSD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。