arXiv:2603.08899cs.CLcs.LG2026-03中稿 · ance

让草稿模型预判生成方向,提升大模型推理速度

ConFu: Contemplate the Future for Better Speculative Sampling

  • 引入未来感知的提示词和软提示,让草稿模型提前获知目标方向
  • 在Llama-3和Qwen-3上提升8%-20%的令牌接受率与生成速度
  • 适合追求高效推理的大模型应用开发者

推测解码通过轻量级草稿模型生成候选标记并由目标模型验证,从而加速大语言模型推理。其效果关键取决于草稿模型质量。现有方法如EAGLE系列虽达顶尖性能,但受限于错误累积:草稿模型仅基于当前前缀预测,导致生成偏离目标模型。本文提出新框架ConFu(预思未来),使草稿模型能预见生成未来方向。ConFu引入(i)可选标记与软提示,以极低开销利用目标模型的未来信号;(ii)基于MoE的动态可选标记机制,实现上下文感知的未来预测;(iii)基于锚定标记采样与未来预测复现的训练框架,学习稳健的未来预测能力。在下游任务中,ConFu相较EAGLE-3,在Llama-3 3B/8B上提升令牌接受率与生成速度8%-11%,在Qwen-3 4B上提升约20%。我们认为本工作首次将推测解码与连续推理标记结合,为加速大模型推理开辟新路径。

原文摘要 · Abstract (English)

Speculative decoding has emerged as a powerful approach to accelerate large language model (LLM) inference by employing lightweight draft models to propose candidate tokens that are subsequently verified by the target model. The effectiveness of this paradigm critically depends on the quality of the draft model. While recent advances such as the EAGLE series achieve state-of-the-art speedup, existing draft models remain limited by error accumulation: they condition only on the current prefix, causing their predictions to drift from the target model over steps. In this work, we propose \textbf{ConFu} (Contemplate the Future), a novel speculative decoding framework that enables draft models to anticipate the future direction of generation. ConFu introduces (i) contemplate tokens and soft prompts that allow the draft model to leverage future-oriented signals from the target model at negligible cost, (ii) a dynamic contemplate token mechanism with MoE to enable context-aware future prediction, and (iii) a training framework with anchor token sampling and future prediction replication that learns robust future prediction. ConFu improves token acceptance rates and generation speed over EAGLE-3 by 8--11\% on Llama-3 3B/8B and by approximately 20\% on Qwen-3 4B across downstream tasks. We believe our work is the first to bridge speculative decoding with continuous reasoning tokens, offering a new direction for accelerating LLM inference.

大模型推理推测解码未来预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。