DSpark通过动态调度验证长度,提升大模型推理速度与系统吞吐。
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

- 采用半自回归结构建模块内依赖,缓解长序列生成中的后缀衰减问题。
- 基于置信度调度验证策略,减少高拒稿风险请求的无效计算,提升系统效率。
- 在真实服务场景中实现60%~85%的生成加速,支持更高交互性要求的性能层级。
推测解码通过将草稿生成与目标验证分离,加速大语言模型推理。尽管近期并行草稿生成器能在单次前向传播中提出长序列,但因缺乏词元间依赖而面临快速接受率下降问题。同时,对所有扩展块无差别验证会浪费批处理资源于高拒绝风险的词元,严重降低高并发服务中的吞吐量。我们提出DSpark,一个统一高吞吐并行生成与自适应负载感知验证的推测解码框架。为保持草稿质量,DSpark采用半自回归架构,结合并行主干与轻量串行模块,引入块内依赖建模,缓解后缀衰减。为优化系统效率,采用置信度调度验证,根据估计的前缀存活概率和引擎特定吞吐特征,动态调整每请求的验证长度。在跨多领域的离线基准测试中,DSpark显著优于现有自回归与并行草稿生成器。部署于DeepSeek-V4实时用户流量环境时,成功缓解验证浪费。相比现有生产基线(MTP-1),DSpark在匹配吞吐水平下,使单用户生成速度提升60%至85%。更重要的是,在严格交互性约束下避免了吞吐量严重下降,实现了此前无法达到的性能层级,推动了服务系统的帕累托前沿升级。
原文摘要 · Abstract (English)
Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。