让语言模型自己规划生成顺序,实现快速并行输出且质量损失小。
Planned Diffusion
- 模型先自回归生成分块计划,再并行去噪各部分。
- 相比传统方法提速1.27至1.81倍,胜率仅下降0.87%到5.4%。
- 适合追求高效率与可调节性能的生成场景。
大多数大语言模型采用自回归方式逐词生成。离散扩散语言模型可并行生成多个词元,但采样需依赖去噪顺序——即每步决定解码哪些词元。现有方法依赖启发式策略,导致质量与延迟间存在陡峭权衡。本文提出计划扩散(Planned Diffusion),训练模型自主决定去噪顺序。该系统使用单一模型,在自回归与扩散生成间切换:首先自回归生成响应的语义独立分块计划;其次并行去噪所有分块。自回归计划使模型能自主定义去噪顺序。在AlpacaEval上,计划扩散相较自回归生成实现1.27倍至1.81倍加速,胜率仅下降0.87%至5.4%,建立新的并行生成帕累托前沿。此外,计划扩散的指令遵循质量随微调计算量增加持续提升,而自回归基线趋于饱和。实现中提供简单运行时控制参数,支持灵活调节质量与延迟平衡。
原文摘要 · Abstract (English)
Most large language models are autoregressive: they generate tokens one at a time. Discrete diffusion language models can generate multiple tokens in parallel, but sampling from them requires a denoising order: a strategy for deciding which tokens to decode at each step. Determining a good denoising order is difficult, and existing approaches use heuristics that create a steep trade-off between quality and latency. We propose planned diffusion, a system that trains the model to determine its own denoising order. Planned diffusion uses a single model that transitions between autoregressive and diffusion-based generation: first, the model autoregressively generates a plan that partitions the response into semantically independent chunks; second, the model denoises all chunks in parallel. The autoregressive plan enables the model to define the denoising order itself. On AlpacaEval, planned diffusion achieves 1.27x to 1.81x speedup over autoregressive generation with only 0.87% to 5.4% drop in win rate, establishing a new Pareto frontier for parallel generation with discrete diffusion. Additionally, planned diffusion's instruction following quality continues to improve with more finetuning compute, while the autoregressive baseline plateaus. Our implementation provides simple runtime knobs that offer tunable control over the quality-latency trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。