arXiv:2509.04474cs.CLcs.AI2025-09被引 6

提出首个评估推测解码加速大模型推理的基准,发现词组方法最有效。

Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling

  • 构建统一实验框架,对比三类推测解码方法。
  • 词组方法在重复性推理中提速显著,优于模型与训练类方法。
  • 适合关注大模型高效推理与重复路径优化的研究者。

测试时扩展已成为通过推理阶段增加计算资源来提升大语言模型(LLMs)推理能力的有力范式。然而,该范式因生成冗余且重复的推理轨迹而存在固有低效问题,导致显著计算开销。推测解码为缓解此问题提供了有前景的方向,但其在结构化、重复性高的测试时扩展场景中的有效性尚未充分探索。为此,我们首次提出了一个全面的基准,用于评估推测解码方法在加速测试时扩展中的表现。该基准在代表性测试时扩展范式(如Best-of-N采样和多轮思考)中提供一致的实验协议,支持对三类主要推测解码方法(基于模型、基于训练、n-gram)的公平比较。大量实验表明,简单的n-gram方法能有效捕捉重复模式,在加速测试时扩展方面展现出独特潜力。这一现象表明,将n-gram方法与基于模型或基于训练的方法结合,可在重复与多样化推理路径间实现加速平衡。我们希望该基准能推动推测解码在测试时扩展领域的进一步研究,从而通过更好处理重复与多样化推理路径,实现更快速、更实用的大模型推理。

原文摘要 · Abstract (English)

Test-time scaling has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs) by allocating additional computational resources during inference. However, this paradigm is inherently inefficient due to the generation of redundant and repetitive reasoning traces, leading to significant computational overhead. Speculative decoding offers a promising avenue for mitigating this inefficiency, yet its efficacy in the structured, repetition-rich context of test-time scaling remains largely unexplored. To bridge this gap, we introduce the first comprehensive benchmark designed to evaluate speculative decoding methods for accelerating LLM test-time scaling. Our benchmark provides consistent experimental protocols across representative test-time scaling paradigms (e.g., Best-of-N sampling and multi-round thinking), enabling a fair comparison of three major categories of speculative decoding: model-based, training-based, and n-gram-based methods. Extensive experiments reveal that simple n-gram-based methods effectively capture repetitive patterns, demonstrating unique potential in accelerating test-time scaling. This phenomenon demonstrates the value of integrating n-gram-based methods with model-based or training-based approaches to balance acceleration for both repetitive and diverse reasoning in test-time scaling. We hope this benchmark spurs further research on speculative decoding for test-time scaling, enabling faster and more practical reasoning in LLMs through better handling of repetitive and diverse reasoning paths.

大模型推理推测解码测试时扩展加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。