arXiv:2601.10064cs.CL2026-01ACL被引 5

通过自适应对齐精炼长链推理,提升小模型数学解题能力

Long-Chain Reasoning Distillation via Adaptive Prefix Alignment

  • 自适应截断教师模型的推理路径,保留有效前缀
  • 在多个数学基准上超越基线3%以上
  • 适合需要高效推理蒸馏的小模型研究者

大语言模型在解决复杂数学问题方面展现出卓越的推理能力。近期研究表明,蒸馏长链推理轨迹可有效提升小规模学生模型的推理性能。然而,教师生成的推理路径往往过长且结构复杂,导致学生模型难以学习,造成监督信号与学习能力之间的不匹配。为此,我们提出前缀对齐蒸馏(P-ALIGN),通过自适应前缀对齐充分挖掘教师思维链(CoT)的潜力。具体而言,P-ALIGN自适应地截断教师推理路径,判断剩余后缀是否简洁且足以引导学生模型;随后利用教师生成的前缀监督学生模型,促进有效前缀对齐。在多个数学推理基准上的实验表明,P-ALIGN性能优于所有基线超过3%。进一步分析显示,P-ALIGN构建的前缀提供了更有效的监督信号,同时避免冗余和不确定推理成分的负面影响。代码已开源:https://github.com/NEUIR/P-ALIGN。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, particularly in solving complex mathematical problems. Recent studies show that distilling long reasoning trajectories can effectively enhance the reasoning performance of small-scale student models. However, teacher-generated reasoning trajectories are often excessively long and structurally complex, making them difficult for student models to learn. This mismatch leads to a gap between the provided supervision signal and the learning capacity of the student model. To address this challenge, we propose Prefix-ALIGNment distillation (P-ALIGN), a framework that fully exploits teacher CoTs for distillation through adaptive prefix alignment. Specifically, P-ALIGN adaptively truncates teacher-generated reasoning trajectories by determining whether the remaining suffix is concise and sufficient to guide the student model. Then, P-ALIGN leverages the teacher-generated prefix to supervise the student model, encouraging effective prefix alignment. Experiments on multiple mathematical reasoning benchmarks demonstrate that P-ALIGN outperforms all baselines by over 3%. Further analysis indicates that the prefixes constructed by P-ALIGN provide more effective supervision signals, while avoiding the negative impact of redundant and uncertain reasoning components. All code is available at https://github.com/NEUIR/P-ALIGN.

推理蒸馏长链推理小模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。