让现有大模型推理更准更快,无需重新训练。
Recirculation

- 在推理阶段引入循环机制,让模型像动态系统一样追踪状态。
- 对Gemma3系列模型,困惑度下降,GSM8k准确率提升21%。
- 无需训练,仅调参即可适配,适合快速部署到现成模型。
我们提出一种面向现成基础模型的推理时架构增强方法,显著降低困惑度并提升生成与推理任务的准确性。该方法在生成阶段几乎不增加延迟,但预填充阶段需串行处理。受前馈变换器中状态更新受限于模型深度的启发,我们的技术——循环回流(recirculation)——引入特定形式的递归,使模型能作为动态系统追踪信念状态。该方法区别于思维链计算(更适合复杂推理)、常见深度递归(循环)以及昂贵的循环变换器训练。我们还提出并评估了一种自适应变体,仅需轻量级超参数调优,冻结原模型权重。相较于现成基线,自适应循环回流在Gemma3系列上取得显著提升:在多个数据集上系统性降低困惑度,在GSM8k上准确率提高21%,并在其他下游任务中实现可靠改进。这一免训练方法通过模型自身反馈指导架构调整,揭示了基于已训练网络特性进行架构演化的可行路径。
原文摘要 · Abstract (English)
We describe an inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across generation and reasoning tasks. Our approach incurs essentially no additional latency during generation, though it requires serial processing in the prefill phase. Motivated by the fundamental limitation that state updates in feedforward transformers are bounded by model depth, our technique, recirculation, introduces a specific form of recurrence that allows the model to act as a dynamical system and track belief states. We distinguish this technique from chain-of-thought computation---which is better reserved for complex inferences rather than basic state tracking---as well as from popular depth-recurrence techniques (looping) and the costly training of recurrent transformers. We also propose and evaluate an adaptive variant of recirculation which requires only light tuning of hyperparameters while freezing the original model weights. Relative to the off-the-shelf baseline, adaptive recirculation achieves remarkable gains on the Gemma3 family, including a systematic reduction in perplexity on a suite of datasets, a 21% increase in accuracy on GSM8k, and reliable improvements in accuracy on other downstream tasks. Our training-free approach succeeds by leveraging the model itself to inform architectural modifications, suggesting a route to architectural evolution guided by a trained network's properties rather than forced, arbitrary design choices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。