一次前向计算生成多个词元,显著提升大模型文本生成速度。
K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modeling

- 通过自洽蒸馏将自回归模型转化为一次性预测多词元的映射。
- 在不同批量下实现2.4至3.5倍加速,质量损失轻微。
- 适合工业级高并发部署场景,兼容现有推理架构。
自回归语言建模是文本生成的主流范式,但逐词元生成导致推理内存受限且效率低下。现有加速方法如推测解码和扩散语言模型虽能提速,却未直接解决高负载批量服务这一工业部署关键问题。本文提出K-Forcing,一种联合预测下一个k个词元的前向传播语言建模范式。K-Forcing将现有自回归模型蒸馏为条件前向映射,将独立均匀噪声转换为多个未来词元的联合采样,仅需一次前向传播。该设计保持固定输出长度,复用自回归教师模型主干,兼容标准自回归推理基础设施。通过渐进式自强制蒸馏训练,逐步扩大预测窗口并使学生模型紧密匹配教师模型的序列分布。在LM1B和OpenWebText上使用标准因果Transformer骨干进行评估,当配置为每前向传播生成k=4个词元时,相比自回归基线,在不同批量下获得约2.4–3.5倍加速,质量下降可接受。随着推理成为现代大模型全生命周期计算成本的主要部分,K-Forcing为真实高负载部署下的自回归生成提供了一条有前景的加速路径。
原文摘要 · Abstract (English)
Autoregressive (AR) language modeling is the dominant paradigm for text generation, yet its sequential token-by-token decoding makes inference memory-bound and inefficient. Existing acceleration approaches, such as speculative decoding and diffusion language models, can yield speedups under certain conditions but do not directly address high-load batch serving--the scenario most critical for industrial-scale deployment. We introduce K-Forcing, a push-forward language modeling paradigm for joint next-k-token decoding. K-Forcing distills an existing AR model into a conditional push-forward mapping--one that transforms independent uniform noise variables into a joint sample of multiple future tokens in a single forward pass. This design preserves fixed-length outputs, reuses the AR teacher backbone, and remains compatible with standard AR serving infrastructure. We train this mapping via progressive self-forcing distillation, which gradually expands the prediction window while enabling the student to closely match the sequence distribution of the AR teacher. We evaluate K-Forcing on LM1B and OpenWebText using a standard causal Transformer backbone. When aggressively configured to generate k = 4 tokens per forward pass, K-Forcing delivers approximately 2.4-3.5x speedup across different batch sizes, while incurring modest quality degradation relative to its AR teacher. As inference increasingly dominates the lifetime compute cost of modern LLMs, K-Forcing offers a promising route toward accelerating AR generation under real-world high-load deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。