无需训练的推理加速方法，显著降低大模型生成延迟。
SSSD: Simply-Scalable Speculative Decoding
- 用n-gram匹配+硬件感知实现无训练推测解码
- 延迟降低最高达2.9倍，性能媲美需训练的方法
- 无需数据准备或调参，适合多语言长文本场景
推测解码已成为加速大语言模型推理的热门技术。然而，现有多数方法在生产环境中的提速效果有限。实现显著加速的方法通常依赖额外训练的草稿模型或辅助组件，增加了部署与维护复杂度。本文提出无需训练的简单可扩展推测解码（SSSD），结合轻量级n-gram匹配与硬件感知推测机制。相比标准自回归解码，SSSD将延迟降低最多2.9倍。其在广泛基准测试中表现与领先训练型方法相当，同时大幅降低采用门槛——无需数据准备、训练或调参，并在语言和领域变化、长上下文场景下表现出更强鲁棒性。
原文摘要 · Abstract (English)
Speculative Decoding has emerged as a popular technique for accelerating inference in Large Language Models. However, most existing approaches yield only modest improvements in production serving systems. Methods that achieve substantial speedups typically rely on an additional trained draft model or auxiliary model components, increasing deployment and maintenance complexity. This added complexity reduces flexibility, particularly when serving workloads shift to tasks, domains, or languages that are not well represented in the draft model's training data. We introduce Simply-Scalable Speculative Decoding (SSSD), a training-free method that combines lightweight n-gram matching with hardware-aware speculation. Relative to standard autoregressive decoding, SSSD reduces latency by up to 2.9x. It achieves performance on par with leading training-based approaches across a broad range of benchmarks, while requiring substantially lower adoption effort--no data preparation, training or tuning are needed--and exhibiting superior robustness under language and domain shift, as well as in long-context settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。