让大模型通过扩散框架动态增加计算量，提升生成精度。
Large Language Models to Diffusion Finetuning
- 用扩散框架微调语言模型，实现测试时可扩展计算。
- 增加扩散步数后准确率持续上升，任务表现显著改善。
- 自适应求解器自动决定所需算力，适合需要灵活推理的场景。
我们提出一种新的微调方法，使预训练的大语言模型（LM）能够通过扩散框架在测试时动态扩展计算量。通过增加扩散步数，我们证明微调后的模型实现了单调递增的准确率，直接转化为下游任务性能的提升。此外，模型可通过强大引导技术精准回答特定主题问题，并利用自适应常微分方程求解器自主判断给定任务所需的计算资源。该方法适用于任何以交叉熵损失预训练的基础模型，不修改其原始权重，完整保留其单步生成能力。实验表明，该方法比传统微调更有效且完全兼容，为统一自回归与扩散框架的优势开辟了正交新方向。
原文摘要 · Abstract (English)
We propose a new finetuning method to provide pre-trained large language models (LMs) the ability to scale test-time compute through the diffusion framework. By increasing the number of diffusion steps, we show our finetuned models achieve monotonically increasing accuracy, directly translating to improved performance across downstream tasks. Furthermore, our finetuned models can expertly answer questions on specific topics by integrating powerful guidance techniques, and autonomously determine the compute required for a given problem by leveraging adaptive ODE solvers. Our method is universally applicable to any foundation model pre-trained with a cross-entropy loss and does not modify any of its original weights, fully preserving its strong single-step generation capabilities. We show our method is more effective and fully compatible with traditional finetuning approaches, introducing an orthogonal new direction to unify the strengths of the autoregressive and diffusion frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。