arXiv:2608.11233cs.CLcs.AI2026-08

给预训练模型加循环深度,让其在低参数下实现更深层推理。

Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets

  • 将模型拆分为前奏、循环块和尾声,通过身份保持路径实现可重用迭代。
  • 在18层深度下仍保持70%准确率,比原模型快7.6倍且保留能力。
  • 适合需要高效深层推理的场景,如复杂逻辑任务或资源受限部署。

一个密集的预训练语言模型可通过引入循环深度,学习持续的隐状态迭代过程,即使仅以最终答案进行评分也能保持性能。将Qwen2.5-0.5B-Instruct拆分为前奏、权重共享的循环块和尾声,并设计了一条保持身份的一次循环路径与后续循环的回连桥。在第1轮循环中,该改造模型在预先注册的ARC基准上不劣于原始模型。三个发现:第一,该机制是可复用的流程而非终值查表,可在600万训练参数(冻结主干)与1.8亿全块两种预算下安装;中间步骤监督下,每轮循环执行一步任务,仅以最终答案评分时仍能保持性能;适配器整体表现接近全块(83.8% vs 84.0%),在深度11内领先,之后落后;口头微调在受控语义生成上达79%-86%(零样本迁移极弱),而基于已安装机制的口头训练比全新训练高18.6个百分点,包括在保留测试集上。第二,该操作可外推至约1.5倍监督深度,在深度18下仍保持70%准确率。第三,相同大小的草稿模型虽在学习范围内匹配循环模型,但超出后迅速崩溃;循环模型整体胜出(84% vs 72%),在深度10后保留53%对比2.5%,且响应速度快7.6倍。因此,迭代式Transformer能在潜空间中比同等或更大规模的微调模型更快完成更深推理。另一任务反向运行揭示了局限:逆向规则可独立学习,但无法在保持原有机制与泛化能力的同时继续掌握,触及灾难性干扰边界。深度选择问题仍待解决。

原文摘要 · Abstract (English)

A dense, pretrained language model can be retrofitted with recurrent depth and learn an iterative latent transition that persists after outcome-only annealing. Qwen2.5-0.5B-Instruct is split into a Prelude, a weight-tied Recurrent Block, and a Coda, with an identity-preserving one-loop path and a re-entry bridge on later loops. At loop 1 the retrofit remains non-inferior to its base on a preregistered ARC battery. Three findings. First, the mechanism is a reusable procedure rather than terminal-answer lookup, and installs at two budgets: 6M trained parameters over frozen base weights and 180M full-block. With intermediate-step supervision, the model computes one task step per loop and persists when only final answers are graded. The adapter matched the full block overall (83.8% versus 84.0%), led through depth 11, and trailed beyond. Verbal fine-tuning reached 79-86% on controlled verbal renderings (zero-shot transfer was minimal), and adapter verbal training begun from the installed mechanism outpaced matched fresh training by 18.6 points, including on a held-out test set. Second, the operation extrapolates to roughly 1.5 times its supervised depth, holding 70% accuracy through depth 18. Third, a same-size scratchpad-trained model matched the recurrent model within its learned horizon but collapsed beyond it. The recurrent model won overall, 84% versus 72%, retained 53% versus 2.5% beyond depth 10, and answered 7.6 times faster. An iterative transformer can therefore perform deeper reasoning in latent space faster than comparable or larger models fine-tuned on the same task, in a system-level comparison. A second task, running the rule in reverse, exposed the limits: the inverse was learnable in isolation, but no continuation acquired it while preserving the installed mechanism and general capability, a catastrophic-interference boundary. Learned depth selection remains open.

循环模型深层推理参数效率推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。