arXiv:2506.04708cs.CL2025-06EMNLP被引 4

不依赖额外模型,用冗余推理模式加速语言模型生成。

Accelerated Test-Time Scaling with Model-Free Speculative Sampling

  • 利用推理路径中的重复模式,无须训练即可预测下一个词。
  • 相比标准自回归解码,延迟降低60%-65%,准确率不变。
  • 适用于任何现成模型,适合需要高效推理的部署场景。

语言模型在推理任务中通过测试时扩展技术(如best-of-N采样和树搜索)展现出强大能力,但这些方法通常消耗大量计算资源,带来性能与效率的权衡。本文提出STAND(STochastic Adaptive N-gram Drafting),一种新颖的无模型推测解码方法,利用推理轨迹中固有的冗余性,在不牺牲准确率的前提下实现显著加速。分析表明,推理路径频繁复用相似推理模式,使得无需独立草稿模型即可高效进行无模型的词元预测。通过引入随机草稿、基于日志概率的内存高效N元组模块,结合优化的Gumbel-Top-K采样和数据驱动的树构建,STAND显著提升了词元接受率。在多个模型和推理任务(AIME-2024、GPQA-Diamond、LiveCodeBench)上的广泛评估显示,相比标准自回归解码,STAND将推理延迟降低了60%-65%,同时保持准确率。此外,无论在单轨迹解码、批量解码还是测试时树搜索等不同推理模式下,STAND均持续优于现有最优推测解码方法。作为无模型方法,STAND可直接应用于任何现有语言模型,无需额外训练,是一种强大的即插即用加速方案。

原文摘要 · Abstract (English)

Language models have demonstrated remarkable capabilities in reasoning tasks through test-time scaling techniques like best-of-N sampling and tree search. However, these approaches often demand substantial computational resources, creating a critical trade-off between performance and efficiency. We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach that exploits the inherent redundancy in reasoning trajectories to achieve significant acceleration without compromising accuracy. Our analysis shows that reasoning paths frequently reuse similar reasoning patterns, enabling efficient model-free token prediction without requiring separate draft models. By introducing stochastic drafting and preserving probabilistic information through a memory-efficient logit-based N-gram module, combined with optimized Gumbel-Top-K sampling and data-driven tree construction, STAND significantly improves token acceptance rates. Extensive evaluations across multiple models and reasoning tasks (AIME-2024, GPQA-Diamond, and LiveCodeBench) demonstrate that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding while maintaining accuracy. Furthermore, STAND consistently outperforms state-of-the-art speculative decoding methods across diverse inference patterns, including single-trajectory decoding, batch decoding, and test-time tree search. As a model-free approach, STAND can be applied to any existing language model without additional training, making it a powerful plug-and-play solution for accelerating language model reasoning.

推理加速无模型推测解码语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。