让小模型自己反思错误，生成更有效的推理教学材料
Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection
- 小模型通过反思自身错误，主动优化大模型的推理过程
- 在多个数学推理任务上提升超2%，逻辑更连贯
- 适合想用小模型实现强推理能力的研究者
大型语言模型（LLMs）在数学问题求解中展现出强大的推理能力。近期，将长链思维（CoTs）中的推理能力蒸馏给小型语言模型（SLMs）成为提升小模型性能的可行路径。现有方法通常将长链CoT作为监督信号，通过有监督微调（SFT）传授推理能力，但这些长链教师往往忽视学生模型的实际水平，限制了推理轨迹的有效利用。为此，我们提出误差感知自我反思框架ORION，通过误差感知反思机制对教师CoT进行优化。ORION使学生模型能基于自身推理错误，主动重构更适配的教师CoT。在多个数学推理基准上的实验表明，ORION在所有基线之上持续提升超过2%。进一步分析显示，ORION生成的CoT具有更高连贯性和逻辑一致性，从而成为更有效的SFT监督信号。代码已开源：https://github.com/NEUIR/ORION.git。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have exhibited strong reasoning capabilities and achieved remarkable performance in mathematical problem-solving tasks. Recently, distilling reasoning ability from long-form Chains-of-Thought (CoTs) has emerged as a promising approach for enhancing Small Language Models (SLMs). Existing studies typically treat SLMs as student models and use long-form CoTs as supervision signals for Supervised Fine-Tuning (SFT) to transfer reasoning ability. However, such long-form CoT teachers are usually unaware of the student model's capacity, which limits the effective utilization of the provided reasoning traces. To overcome this limitation, we propose errOr-aware self-ReflectION (ORION), a framework that refines teacher CoTs through an Error-Aware Reflection process. ORION enables the student model to construct more tailored teacher CoTs by refining teacher CoTs and incorporating its own reasoning errors. Experiments on multiple mathematical reasoning benchmarks demonstrate that ORION consistently improves performance by more than 2% over all baselines. Further analysis reveals that the CoTs constructed by ORION exhibit higher coherence and logical consistency, thereby serving as more effective supervision signals for SFT. All codes are available at https://github.com/NEUIR/ORION.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。