arXiv:2607.07409cs.CL2026-07被引 1

提出新方法提升大模型推理速度,无需重训练即可实现高效并行生成。

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

论文配图:DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting
图 1 · 摘自论文原文
  • 分离长短期上下文,用轻量头独立训练实现快速生成。
  • 在数学、代码、对话任务上平均加速比和接受长度均优于DFlash。
  • 无需联合训练,适配多种模型,部署灵活且成本极低。

推测解码通过并行起草多个词元并验证来加速大语言模型推理。块并行起草方法如DFlash通过一次预测整个块提升效率,但其位置级预测缺乏显式的块内因果约束。近期方法如Domino和DSpark尝试引入此类因果性,但需从头训练起草模型,限制灵活性并增加训练成本。本文提出DeLS-Spec,一种解耦长短期上下文的推测解码方法。将固定DFlash模型作为长上下文专家,引入轻量级局部头作为短上下文专家。局部头可独立使用标准下一个词元预测目标训练,无需与目标模型或DFlash主干联合训练,训练成本极低。推理时,融合长短期上下文得分,且局部头不绑定特定DFlash检查点,方法更模块化、灵活。在Qwen3模型上的实验表明,DeLS-Spec在数学、代码和对话基准上持续优于DFlash,提升加速比和平均接受长度。

原文摘要 · Abstract (English)

Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position-wise predictions lack explicit intra-block causal conditioning. Recent methods such as Domino and DSpark attempt to introduce such causality into block-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost. We propose DeLS-Spec, a decoupled long-short context speculative decoding method. DeLS-Spec treats the fixed DFlash model as a long-context expert and introduces a lightweight local head as a short-context expert. The local head can be trained independently with a standard next-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost. At inference time, DeLS-Spec combines long-context and short-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks.

大模型推理推测解码高效生成模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。