让大模型一次跳着预测多个不相邻的词，提速推理并提升长距离依赖捕捉能力
L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models
- 通过跳跃机制跳过中间词，一次预测非连续多个词
- 在多个基准上同时提升模型性能和推理速度，最快提速3.2倍
- 适合追求高效推理的大模型应用，如实时对话系统
大语言模型（LLMs）已取得显著进展。尽管如此，主流的下一个词预测（NTP）方法因固有的序列化过程，在上下文覆盖范围和推理效率上仍受限制。为此，我们提出一种新型的跳跃式多词预测（L-MTP）方法，通过引入跳跃机制扩展多词预测（MTP）的能力。与传统MTP仅生成相邻位置多个词不同，L-MTP有策略地跳过中间词，一次性预测非连续位置的多个词。这种结构化跳跃不仅增强了模型捕捉长距离依赖的能力，还支持专为非连续跳跃生成优化的解码策略，显著加速推理。我们从理论上证明了L-MTP在提升推理效率方面的优势。在多种基准上的实验验证了其在提升模型性能和推理速度方面的有效性。源代码已开源：https://github.com/Xiaohao-Liu/L-MTP。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved notable progress. Despite their success, next-token prediction (NTP), the dominant method for LLM training and inference, is constrained in both contextual coverage and inference efficiency due to its inherently sequential process. To overcome these challenges, we propose leap multi-token prediction~(L-MTP), an innovative token prediction method that extends the capabilities of multi-token prediction (MTP) by introducing a leap-based mechanism. Unlike conventional MTP, which generates multiple tokens at adjacent positions, L-MTP strategically skips over intermediate tokens, predicting non-sequential ones in a single forward pass. This structured leap not only enhances the model's ability to capture long-range dependencies but also enables a decoding strategy specially optimized for non-sequential leap token generation, effectively accelerating inference. We theoretically demonstrate the benefit of L-MTP in improving inference efficiency. Experiments across diverse benchmarks validate its merit in boosting both LLM performance and inference speed. The source code is available at https://github.com/Xiaohao-Liu/L-MTP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。