一次前向传播生成多个词元，显著提升语言模型推理速度。
Parallel Token Prediction for Language Models
- 将随机性引入输入变量，使多词元可同时确定预测
- 单次调用实现任意词元间依赖关系，实测提速2.4倍
- 无需教师模型，支持蒸馏或逆自回归训练，适用广泛
语言模型的自回归解码本质上缓慢，每次前向传播仅生成一个词元。我们提出并行词元预测（PTP），一种通用框架，可在单次模型调用中预测多个词元。PTP将随机性的来源从采样阶段转移到输入变量，使未来词元成为这些输入的确定性函数，从而在单次前向传播中联合预测。我们证明，单次PTP调用可表示任意词元间依赖关系。PTP可通过蒸馏现有模型或通过无教师的逆自回归训练进行训练。实验表明，PTP在多样化任务的推测解码基准上实现了2.4倍的速度提升。代码与检查点见https://github.com/mandt-lab/ptp。
原文摘要 · Abstract (English)
Autoregressive decoding in language models is inherently slow, generating only one token per forward pass. We propose Parallel Token Prediction (PTP), a general-purpose framework for predicting multiple tokens in a single model call. PTP moves the source of randomness from post-hoc sampling to random input variables, making future tokens deterministic functions of those inputs and thus jointly predictable in a single forward pass. We prove that a single PTP call can represent arbitrary dependencies between tokens. PTP is trained by distilling an existing model or through inverse autoregressive training without a teacher. Experimentally, PTP achieves a 2.4x speedup on a diverse-task speculative decoding benchmark. We provide code and checkpoints at https://github.com/mandt-lab/ptp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。