让大模型推理更一致,小模型学得更快更准。
Structural Rationale Distillation via Reasoning Space Compression

- 用动态路径库约束大模型推理结构,提升一致性。
- 在5个基准上超越主流方法,且用更少token完成。
- 适合需要稳定推理模式的场景,如数学与常识推理。
在将大型语言模型(LLMs)的推理能力蒸馏给小型模型时,教师模型对相似问题的推理路径常结构迥异、策略多变,导致学生模型面临噪声严重的监督信号。本文提出通过推理路径压缩进行蒸馏(D-RPC),强制教师从一个动态维护的紧凑可复用高阶推理路径库中选择路径。针对每个训练问题,D-RPC检索最相关路径并引导教师遵循,生成跨相似问题具有一致性但覆盖多样题型的推理过程。基于PAC-Bayes的分析形式化了路径库大小与覆盖范围之间的权衡:较小库降低监督熵但可能遗漏覆盖,而泛化界指明了最优中间规模,经消融实验验证。在两个学生模型上,于五个数学与常识推理基准上,D-RPC持续优于链式思维蒸馏、自由生成推理、直接蒸馏及结构化监督基线,且相比模板密集方法使用更少的词元。
原文摘要 · Abstract (English)
When distilling reasoning from large language models (LLMs) into smaller ones, teacher rationales for similar problems often vary wildly in structure and strategy. Like a chef who makes the same dish differently each time, this inconsistency burdens the student with noisy supervision that is hard to internalize. We propose Distillation through Reasoning Path Compression (D-RPC), which constrains the teacher to follow a compact, dynamically maintained bank of reusable high-level reasoning paths. For each training question, D-RPC retrieves the most relevant path and conditions the teacher to follow it, producing rationales that are consistent across similar problems yet diverse enough to cover different problem types. A PAC-Bayes analysis formalizes the resulting trade-off between bank size and coverage: smaller banks reduce supervision entropy but risk coverage gaps, and the generalization bound identifies an optimal intermediate size confirmed by our ablations. Across five math and commonsense reasoning benchmarks with two student models, D-RPC consistently outperforms chain-of-thought distillation, freeform rationale generation, direct distillation, and structured-supervision baselines, while using fewer tokens than template-heavy alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。