让大模型并行预测多个词，显著降低推理延迟。
P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
- 用共享隐藏状态实现多词并行生成，替代逐词预测。
- 在长文本上训练速度提升1.10-1.36倍，支持超长上下文。
- 适合需要低延迟推理的工业级大模型应用。
推理型大模型生成更长输出，需在长序列上训练推测解码器。并行起草——每前向传播预测多个词——相比逐词生成可降低延迟，但训练复杂度随序列长度与并行位置数乘积呈平方增长，使长上下文训练不可行。我们提出P-EAGLE，通过可学习共享隐藏状态将EAGLE从自回归转为并行多词预测。为实现长上下文训练，我们设计了注意力掩码预计算与序列分块技术，支持在单个序列内进行梯度累积以完成并行预测训练。我们在vLLM中实现P-EAGLE，相较于自回归EAGLE-3，在GPT-OSS 120B、20B及Qwen3-Coder 30B上实现1.10–1.36倍加速。
原文摘要 · Abstract (English)
Reasoning LLMs produce longer outputs, requiring speculative decoding drafters trained on extended sequences. Parallel drafting - predicting multiple tokens per forward pass - offers latency benefits over sequential generation, but training complexity scales quadratically with the product of sequence length and parallel positions, rendering long-context training impractical. We present P(arallel)-EAGLE, which transforms EAGLE from autoregressive to parallel multi-token prediction via a learnable shared hidden state. To scale training to long contexts, we develop a framework featuring attention mask pre-computation and sequence partitioning techniques, enabling gradient accumulation within individual sequences for parallel-prediction training. We implement P-EAGLE in vLLM and demonstrate speedups of 1.10-1.36x over autoregressive EAGLE-3 across GPT-OSS 120B, 20B, and Qwen3-Coder 30B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。