让模型自适应决定何时暂停,用更聪明的计算提升推理效果。
Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production
- 通过特殊<don't know>标记动态触发暂停,自主控制计算步数。
- 在多个数据集上降低困惑度,下游任务准确率显著提升。
- 无需额外算力,适用于预训练和微调阶段,适合高精度推理场景。
在基础模型推理时缩放方法中,基于宽度的缩放策略通过在输入流中插入<pause>标记来延迟模型输出,具有提升模型表达力且训练与推理高度并行的优势。现有文献中训练模型使用<pause>标记的方法依赖标准交叉熵目标,仅在暂停序列的最终步骤读取模型输出,缺乏模型自我调节处理或响应就绪信号的机制,将额外计算步骤视为静态障碍而非可动态利用的资源。本文提出一种监督损失函数Catch Your Breath(CYB),将其建模为序列决策问题,训练模型动态自主地为每个输入词元调整计算步数。模型通过输出特殊<don't know>标记来表明需要更多计算步骤,并通过暂停延迟响应;可多次拒绝以获得更长延迟。实验表明,无论在预训练还是微调阶段引入,CYB均显著优于标准交叉熵,降低困惑度并提升下游准确率,且无额外计算或内存开销。
原文摘要 · Abstract (English)
Within the landscape of inference-time scaling methods for foundation models, a width-based approach to scaling -- which involves the insertion of <pause> tokens in the input stream to delay model responses -- offers a unique advantage by increasing model expressivity while remaining highly parallelizable at both training and inference. The existing literature on training models to utilize <pause> tokens relies on the standard cross-entropy objective in which the model output is read out and evaluated only at the final step of a pause sequence. This approach provides no mechanism for the model to regulate its own processing or to signal readiness to respond, treating the additional compute steps as a static barrier rather than a resource to be used adaptively. We propose a supervised loss, Catch Your Breath (CYB), framed as a sequential-decision problem, that trains a model to dynamically and autonomously scale the number of compute steps used for each input token. The model indicates the need for additional compute steps by emitting a special <don't know> output, delaying its response via a pause. The model can abstain multiple times to obtain longer delays. Our experiments demonstrate that CYB significantly outperforms standard cross-entropy when introduced either in pretraining or fine-tuning, reducing perplexity and enhancing downstream accuracy with no additional computational or memory cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。