让大模型在预训练阶段就学会循环推理，效果媲美120亿参数模型。
Scaling Latent Reasoning via Looped Language Models
- 用潜空间迭代计算和熵正则目标，把推理能力嵌入预训练
- 77亿参数模型在多个任务上超越普通模型，接近120亿参数水平
- 推理过程更贴近最终答案，适合需要精准逻辑的场景
现代大模型主要通过链式思维（CoT）等显式文本生成方式进行推理，这将思考过程推迟到微调阶段，未能充分利用预训练数据。我们提出并开源了名为Ouro的循环语言模型（LoopLM）家族，其命名源自自噬蛇意象，通过（i）潜空间中的迭代计算，（ii）基于熵正则的目标实现学习深度分配，（iii）扩展至770亿参数量级的预训练数据进行训练。Ouro 1.4B与2.6B模型在多种基准测试中表现优异，性能相当于高达120亿参数的先进模型。受控实验表明，这种优势并非源于知识容量增加，而是源于更强的知识操作能力。此外，与显式链式思维相比，LoopLM生成的推理轨迹与最终输出更一致。我们希望这些结果展示出一种面向推理时代的新型可扩展路径。模型代码与资源见：http://ouro-llm.github.io。
原文摘要 · Abstract (English)
Modern LLMs are trained to "think" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. Through controlled experiments, we show this advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities. We also show that LoopLM yields reasoning traces more aligned with final outputs than explicit CoT. We hope our results show the potential of LoopLM as a novel scaling direction in the reasoning era. Our model is available here: http://ouro-llm.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。