无需训练即可并行预测多个词元，提升生成效率与速度。
Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing
- 用嵌入空间中随机采样的掩码词元探测模型，实现无训练多词元预测。
- 在LLaMA3上比基线提升7-11%的接受长度，吞吐量最高提升19%。
- 适合追求高效推理、不希望微调或引入额外模型的开发者使用。
大型语言模型（LLMs）虽仅训练为单步词元预测，却具备潜在的多词元预测能力。我们提出ESP（Embedding-Space Probing），一种简单且无需训练的MTP方法：通过从模型嵌入空间中实时采样掩码词元进行探测，实现无需修改权重或依赖草稿模型的并行未来词元预测。ESP通过从掩码词元的词元分布中采样Top-K候选，构建推测词元树，并采用轻量剪枝规则保留高概率延续路径。生成时并行验证预测结果，实现无损解码，显著减少模型调用次数，提升词元吞吐量。ESP在所有测试基准上均优于现有无训练基线，在LLaMA3上比LADE提升7-11%接受长度，在Qwen3上提升7-8%；吞吐量最高提升15-19%。最后，我们提供理论分析与实证证据，表明解码器层自然将掩码词元表示对齐至下一词元状态，从而实现无需重训练或辅助模型的准确多步预测。
原文摘要 · Abstract (English)
Large Language Models (LLMs) possess latent multi-token prediction (MTP) abilities despite being trained only for next-token generation. We introduce ESP (Embedding-Space Probing), a simple and training-free MTP method that probes an LLM using on-the-fly mask tokens drawn from its embedding space, enabling parallel future-token prediction without modifying weights or relying on draft models. ESP constructs a speculative token tree by sampling Top-K candidates from mask-token logits and applies a lightweight pruning rule to retain high-probability continuations. During generation, predictions are verified in parallel, yielding lossless decoding while significantly reducing model calls and increasing token throughput. ESP consistently outperforms existing training-free baselines, improving acceptance length by 7-11% over LADE on LLaMA3 and 7-8% on Qwen3, and increasing throughput by up to 15-19% over the strongest baseline. Finally, we provide theoretical insight and empirical evidence showing that decoder layers naturally align mask-token representations with next-token states, enabling accurate multi-step prediction without retraining or auxiliary models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。