将推测解码扩展到多设备协同，提升边缘云环境下的大模型推理速度。
DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving
- 通过多设备协同生成与验证，实现分布式推测解码。
- 在多种负载下最高提速1.1倍，吞吐率提升9.7%。
- 适合需要快速部署、弹性扩展的大模型服务场景。
大语言模型（LLM）推理常因解码延迟高、异构边缘-云环境中可扩展性差而受限。现有推测解码（SD）技术虽能加速分词生成，但仅限单机执行。本文提出DSD，一种通过协调草稿-目标执行的分布式推测解码框架，首次支持多设备部署。针对该范式缺乏前人研究，我们设计DSD-Sim，一个离散事件模拟器，捕捉网络、批处理与调度动态。基于模拟洞察，进一步提出自适应窗口控制（AWC）策略，动态调节推测窗口大小以优化吞吐。跨多种工作负载的实验表明，DSD相较现有SD基线最高实现1.1倍加速，吞吐率提升9.7%，实现了边缘与云端敏捷、可扩展的LLM服务。
原文摘要 · Abstract (English)
Large language model (LLM) inference often suffers from high decoding latency and limited scalability across heterogeneous edge-cloud environments. Existing speculative decoding (SD) techniques accelerate token generation but remain confined to single-node execution. We propose DSD, a distributed speculative decoding framework that extends SD to multi-device deployments through coordinated draft-target execution. Given the lack of prior work on simulating this paradigm, we first introduce DSD-Sim, a discrete-event simulator that captures network, batching, and scheduling dynamics. Building on insights from DSD-Sim, we further design an Adaptive Window Control (AWC) policy that dynamically adjusts speculation window size to optimize throughput. Experiments across diverse workloads show that DSD achieves up to 1.1x speedup and 9.7% higher throughput over existing SD baselines, enabling agile and scalable LLM serving across edge and cloud.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。