让语言模型在生成时反复思考，提升推理能力。
PonderLM: Pretraining Language Models to Ponder in Continuous Space
- 通过多次前向传播生成加权嵌入，实现连续空间中的持续思考。
- PonderPythia-2.8B性能超越6.9B原版，接近12B模型。
- 无需人工标注，适用于多种主流模型架构，适合追求高效推理的开发者。
人类在表达复杂语句前会进行深思，这种专注处理有助于提升认知质量。本文将这一‘思考’过程引入语言模型：在单个词元生成步骤中，重复调用前向传播，不直接采样输出词元，而是根据预测分布生成所有词元嵌入的加权和作为新输入。该嵌入被反馈回模型以进行下一次前向计算，形成持续思考循环。我们证明，仅通过自监督学习，模型即可学会此过程，无需任何人工标注。在GPT-2、Pythia和LLaMA三种主流开源架构上验证，涵盖9个下游任务，结果表明该方法具备广泛有效性。特别地，增强后的PonderPythia模型表现优异：PonderPythia-2.8B超越官方Pythia-6.9B，媲美Pythia-12B；PonderPythia-1B达到TinyLlama-1.1B水平，后者训练数据量为前者的10倍。代码已公开于https://github.com/LUMIA-Group/PonderingLM。
原文摘要 · Abstract (English)
Humans ponder before articulating complex sentence elements, enabling deeper cognitive processing through focused effort. In this work, we introduce this pondering process into language models by repeatedly invoking the forward process within a single token generation step. During pondering, instead of generating an actual token sampled from the prediction distribution, the model ponders by yielding a weighted sum of all token embeddings according to the predicted token distribution. The generated embedding is then fed back as input for another forward pass. We show that the model can learn to ponder in this way through self-supervised learning, without any human annotations. Experiments across three widely used open-source architectures-GPT-2, Pythia, and LLaMA-and extensive downstream task evaluations demonstrate the effectiveness and generality of our method. On 9 downstream benchmarks, our pondering-enhanced Pythia models significantly outperform the official Pythia models. Notably, our PonderPythia models demonstrate remarkable effectiveness: PonderPythia-2.8B surpasses Pythia-6.9B and rivals Pythia-12B, while our PonderPythia-1B matches TinyLlama-1.1B, a model trained on 10 times more data. The code is available at https://github.com/LUMIA-Group/PonderingLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。