让小模型学会像大模型一样推理，还能自动找适合自己的路。
Beyond Templates: Dynamic Adaptation of Reasoning Demonstrations via Feasibility-Aware Exploration
- 根据小模型能力动态调整模仿策略，避免硬套大模型步骤。
- 在多个任务上提升小模型表现，比传统微调更省数据、泛化更强。
- 适合资源受限场景，如移动端或边缘设备部署小模型。
大型语言模型（LLMs）展现出强大的推理能力，但将其推理能力对齐到小型语言模型（SLMs）仍面临分布差异和模型容量有限的挑战。现有推理数据集多为强大LLMs设计，直接用于弱模型时性能下降明显。本文提出动态推理轨迹适配框架DART，弥合专家推理轨迹与多样小模型间的差距。DART不统一模仿专家步骤，而是通过解题模拟评估每步的可适配性，当专家步骤超出学生模型能力（即出现仿效缺口）时，学生自主探索替代推理路径，同时保证结果一致性。我们在多个推理基准和模型规模上验证DART，结果表明其显著提升泛化能力和数据效率，优于静态微调。该方法通过将训练信号与学生推理能力对齐，提升监督质量，为资源受限模型提供可扩展的推理对齐方案。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown remarkable reasoning capabilities, yet aligning such abilities to small language models (SLMs) remains a challenge due to distributional mismatches and limited model capacity. Existing reasoning datasets, typically designed for powerful LLMs, often lead to degraded performance when directly applied to weaker models. In this work, we introduce Dynamic Adaptation of Reasoning Trajectories (DART), a novel data adaptation framework that bridges the capability gap between expert reasoning trajectories and diverse SLMs. Instead of uniformly imitating expert steps, DART employs a selective imitation strategy guided by step-wise adaptability estimation via solution simulation. When expert steps surpass the student's capacity -- signaled by an Imitation Gap -- the student autonomously explores alternative reasoning paths, constrained by outcome consistency. We validate DART across multiple reasoning benchmarks and model scales, demonstrating that it significantly improves generalization and data efficiency over static fine-tuning. Our method enhances supervision quality by aligning training signals with the student's reasoning capabilities, offering a scalable solution for reasoning alignment in resource-constrained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。