用思维深度排序训练任务，让大模型逐步攻克更复杂的推理题。
Reasoning Steps as Curriculum: Using Depth of Thought as a Difficulty Signal for Tuning LLMs
- 以思维步骤数衡量难度，按由浅入深顺序组织训练数据。
- 在相同预算下，该方法比长度或评分排序更有效提升模型表现。
- 适用于各类教师模型，只需简单格式控制即可保持效果一致。
大语言模型的课程学习需要与推理能力对齐且可扩展、可解释的难度信号。本文提出一个简单假设：人类需要更深层次思考的任务，对模型也应更难。因此，我们将难度定义为思维深度（DoT），通过统计教师模型推理轨迹中的离散步骤数（如思维链）来量化。随后，我们按此DoT值从浅到深排列训练任务，并设计了大规模推导、验证和调度该课程的方法。研究提出三个可验证假设：(i) DoT与传统推理基准上的难度高度相关；(ii) 在相同训练预算下，基于DoT的课程优于基于长度或人工评分的课程；(iii) 难度指标在不同教师模型间具有鲁棒性，仅需轻量级格式控制。我们构建了评估框架，讨论了潜在威胁（如教师风格、长度混淆）及缓解策略。整体目标是推动面向推理任务的、具认知基础且可解释的课程学习范式。
原文摘要 · Abstract (English)
Curriculum learning for training LLMs requires a difficulty signal that aligns with reasoning while remaining scalable and interpretable. We propose a simple premise: tasks that demand deeper depth of thought for humans should also be harder for models. Accordingly, we define difficulty as depth of thought (DoT) and operationalize it by counting the discrete steps in a teacher model's reasoning trace (e.g., Chain-of-Thought). We then train with a shallow to deep curriculum ordered by this DoT and outline how to derive, validate, and schedule it at scale. Our position yields three testable hypotheses: (i) DoT correlates with conventional difficulty on reasoning benchmarks, (ii) DoT-ordered curricula outperform length- or judge-scored curricula under matched budgets, and (iii) the difficulty is robust across teacher models given light formatting controls. We propose an evaluation framework and discuss threats to validity (teacher style, length confounds) alongside practical mitigations. Taken together, we aim to move toward cognitively grounded, interpretable curricula for reasoning-centric training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。