arXiv:2605.24793cs.CL2026-05

让小模型与大模型协作,提升推理速度和准确率。

Beyond the Target: From Imitation to Collaboration in Speculative Decoding

论文配图:Beyond the Target: From Imitation to Collaboration in Speculative Decoding
图 1 · 摘自论文原文
  • 用强化学习训练仲裁策略,动态决定采纳小模型或大模型的预测。
  • 在部分分歧场景下采纳小模型输出,使最终答案正确率超过仅用大模型。
  • 适合追求高推理效率且需保持高准确率的AI系统部署场景。

推测解码(SPD)通过较小的草稿模型预生成多个未来词元,并由较大的目标模型并行验证,从而加速大语言模型的推理。主流方法将目标模型视为唯一可靠的教师,仅当草稿词元与目标预测完全一致时才接受。该设计隐含假设:目标模型在每个位置都更优,但实践中并不成立。尽管草稿模型整体较弱,但在部分词元层面表现更优。实验发现,在草稿与目标不一致的情况下,草稿的选择常能导向正确最终答案。为此,本文提出协同推测解码(CoSpec),不再将目标模型视为唯一的词元级权威。CoSpec通过强化学习训练仲裁策略,判断在分歧时是否采纳草稿模型输出,若此举更可能获得正确结果则选择草稿。实验表明,CoSpec在保持显著加速的同时,性能超越仅使用目标模型的方法。该方法从模仿转向协作,为推测解码提供了新视角。

原文摘要 · Abstract (English)

Speculative decoding (SPD) accelerates large language model (LLM) inference by letting a smaller draft model propose multiple future tokens that are verified in parallel by a larger target model. The dominant SPD paradigm treats the target model as the sole reliable teacher, accepting a draft token only when it exactly matches the target prediction. This design implicitly assumes that the target is always the better choice at every position. In practice, this assumption does not hold. Although the draft is the weaker model overall, it is not uniformly inferior at the token level. In a meaningful fraction of cases where draft and target disagree, the draft's choice is the one that leads to the correct final answer. Inspired by this, we introduce \textbf{Collaborative Speculative Decoding (CoSpec)}, a generalization of SPD that no longer treats the target model as the sole token-level authority. CoSpec trains an arbitration policy via reinforcement learning to decide whether to accept tokens from the draft or target model, selectively accepting draft tokens at mismatches when doing so is likely to yield a correct final answer. Experimental results show that CoSpec maintains substantial speedups while surpassing target-only performance. By shifting the emphasis from imitation to collaboration, CoSpec suggests a new perspective on speculative decoding.

推理加速协同解码强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。