让小模型学会大模型的复杂推理，突破传统方法的天花板。
HEAL: Hindsight Entropy-Assisted Learning for Reasoning Distillation
- 用熵动态检测推理断点，主动注入事后提示修复错误路径。
- 在多个基准上超越标准微调，推理准确率显著提升。
- 适合需要高效知识迁移的模型压缩场景，尤其对难例有效。
将大型推理模型（LRMs）的推理能力蒸馏到小型模型，常受限于拒绝采样的缺陷。传统方法将教师视为静态过滤器，丢弃教师无法独立解决的复杂“边缘案例”，从而人为制造了学生学习的“教师天花板”。本文提出无需强化学习的HEAL框架，借鉴最近发展区（ZPD）教育理论，融合三个核心模块：(1) 有指导的熵辅助修复（GEAR），通过熵动态检测关键推理断点，并注入针对性的事后提示修复失败轨迹；(2) 糟糕困惑度-不确定性比估计器（PURE），基于比率的过滤机制，降低高异常性捷径式推理路径的比例；(3) 逐步答案引导式课程演化（PACE），三阶段蒸馏策略，从基础对齐逐步过渡到难题适应。在多个基准上的大量实验表明，HEAL显著优于传统SFT蒸馏及其他基线方法。
原文摘要 · Abstract (English)
Distilling reasoning capabilities from Large Reasoning Models (LRMs) into smaller models is typically constrained by the limitations of rejection sampling. Standard methods treat the teacher as a static filter, discarding complex "corner-case" problems where the teacher fails to explore valid solutions independently, thereby creating an artificial "Teacher Ceiling" for the student. In this work, we propose Hindsight Entropy-Assisted Learning (HEAL), an RL-free framework designed to bridge this reasoning gap. Drawing on the educational theory of the Zone of Proximal Development (ZPD), HEAL synergizes three core modules: (1) Guided Entropy-Assisted Repair (GEAR), an active intervention mechanism that detects critical reasoning breakpoints via entropy dynamics and injects targeted hindsight hints to repair broken trajectories; (2) Perplexity-Uncertainty Ratio Estimator (PURE), a ratio-based filtering heuristic that reduces high-anomaly shortcut-like rationales; and (3) Progressive Answer-guided Curriculum Evolution (PACE), a three-stage distillation strategy that organizes training from foundational alignment to hard-case adaptation. Extensive experiments on multiple benchmarks demonstrate that HEAL significantly outperforms traditional SFT distillation and other baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。