让预训练大模型通过循环推理提升推理能力,无需重新训练
LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models

- 将预训练模型拆解为编码器、循环推理块和解码器
- 在多个基准上实现显著推理性能提升,且不增加生成长度
- 适合希望低成本增强现有模型推理能力的研究者
循环计算在提升大语言模型的推理性能方面展现出潜力,可通过扩展测试时计算量实现。然而,现有方法通常需要从头训练循环模型或进行破坏性改造,带来高昂计算成本并可能损害预训练能力。为此,我们提出「环路深度升维」(LoopUS),一种后训练框架,可将标准预训练大模型转换为循环架构。其核心技术是将预训练模型重构为编码器、循环推理模块和解码器。通过四个核心组件实现:(1) 基于阶段表征动态的模块分解;(2) 输入依赖的可选门控机制,缓解隐藏状态漂移;(3) 随机深层监督,实现长递归周期下的高效学习;(4) 置信度头,支持自适应提前退出。这些机制共同使非循环模型稳定地实现循环推理,提升推理表现,同时避免生成轨迹延长或从头训练。详细信息见 https://thrillcrazyer.github.io/LoopUS
原文摘要 · Abstract (English)
Looped computation shows promise in improving the reasoning-oriented performance of LLMs by scaling test-time compute. However, existing approaches typically require either training recurrent models from scratch or applying disruptive retrofits, which involve substantial computational costs and may compromise pretrained capabilities. To address these limitations, we introduce \textbf{Looped Depth Up-Scaling} (LoopUS), a post-training framework that converts a standard pretrained LLM into a looped architecture. As a key technical contribution, LoopUS recasts the pretrained LLM into an encoder, a looped reasoning block, and a decoder. It operationalizes this latent-refinement architecture through four core components: (1) block decomposition, guided by staged representation dynamics; (2) an input-dependent selective gate to mitigate hidden-state drift; (3) random deep supervision for memory-efficient learning over long recursive horizons; and (4) a confidence head for adaptive early exiting. Collectively, these mechanisms transform a standard non-looped model into a looped form while stabilizing it against both computational bottlenecks and representation collapse. Through stable latent looping, LoopUS improves reasoning-oriented performance without extending the generated traces or requiring recurrent training from scratch. For more details, see https://thrillcrazyer.github.io/LoopUS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。