通过动态调整递归深度，让小型递归模型训练快1.7倍且更稳定。
Accelerating Training Speed of Tiny Recursive Models with Curriculum Guided Adaptive Recursion
- 用分阶段递增的递归深度和衰减监督权重，优化训练过程
- 在极难题目上训练时间缩短至6.38小时，准确率仅降0.63%
- 适合追求高效推理的神经符号系统与程序生成应用
递归推理模型通过迭代精炼实现强性能，使小模型媲美大语言模型。但训练成本高，常需36 GPU小时完成极难题目。现有方法固定递归深度且统一监督权重，导致效率低下。本文提出CGAR（课程引导自适应递归），引入渐进深度课程（PDC）动态调节递归深度，以及分层监督权重（HSW）对各步骤施加指数衰减重要性。PDC采用三阶段调度，从浅层（2,1）过渡到完整深度（6,3），减少41.4%计算量；HSW使梯度方差降低40%，加速收敛。在Sudoku-Extreme上，CGAR实现1.71倍训练提速（10.93→6.38小时），准确率仅下降0.63%（86.65%→86.02%）。仅用PDC即达2.26倍提速，准确率85.47%；HSW贡献1.61倍提速。训练模型推理效率更高，100%停止准确率，推理步数减少11%。结论：通过将深度作为可调度参数，CGAR显著降低资源消耗，防止过拟合，使递归模型在中等硬件上实用化，适用于神经符号人工智能与程序合成。
原文摘要 · Abstract (English)
Background: Recursive reasoning models achieve strong performance through iterative refinement, allowing small networks to match large language models. However, training is computationally expensive, often requiring 36 GPU-hours for Sudoku extreme. Existing models use fixed recursion depth and uniform supervision weighting, leading to inefficient training. Objectives: We propose CGAR (Curriculum-Guided Adaptive Recursion), applying curriculum learning to architectural depth. CGAR introduces Progressive Depth Curriculum (PDC) to dynamically adjust recursion depth and Hierarchical Supervision Weighting (HSW) to apply exponentially decaying importance to supervision steps. Methods: PDC implements a three-stage schedule transitioning from shallow (2, 1) to full depth (6, 3) configurations, providing 41.4% FLOPs reduction. HSW applies exponential decay to supervision steps, achieving 40% gradient variance reduction and accelerated convergence. Results: On Sudoku-Extreme, CGAR achieves 1.71x training speedup (10.93 to 6.38 hours) with only a 0.63% accuracy drop (86.65% to 86.02%). PDC alone achieves 2.26x speedup with 85.47% accuracy, showing a Pareto improvement in efficiency and quality. HSW provides 1.61x speedup. CGAR-trained models show superior inference efficiency with 100% halting accuracy and 11% fewer reasoning steps. Conclusions: CGAR enables efficient training of recursive models on modest hardware. By treating depth as a scheduled parameter, it achieves substantial savings and prevents overfitting, making these models practical for neurosymbolic AI and program synthesis. https://github.com/Kaleemullahqasim/CGAR and huggingface.co/Kaleemullah/trm-cgar-sudoku.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。