arXiv:2512.13194cs.CLcs.AI2025-12被引 1

动态调整采样阈值,减少大模型生成中的随机拒稿,提升推理速度。

Efficient Adaptive Rejection Sampling for Accelerating Speculative Decoding in Large Language Models

  • 根据目标模型的预测不确定性动态调节接受阈值
  • 在GSM8K上实现18.12%吞吐量提升,准确率仅降0.84%
  • 无需修改模型结构,可无缝接入现有推理框架

推测解码是通过快速草稿模型生成候选词序列,并由大型目标模型并行验证来加速大语言模型自回归推理的主流技术。然而,其核心组件——拒绝采样机制——依赖于固定且与上下文无关的随机阈值,导致在高不确定性生成场景中频繁出现合理候选词被随机拒绝的问题,严重降低推理效率。本文提出高效自适应拒绝采样(EARS),通过引入目标模型自身预测不确定性的度量(1 - max(P_target))作为容忍度项,动态调整接受阈值。当模型不确定时智能放宽标准,显著减少随机拒稿;当模型自信时仍保持严格标准。在创意写作和开放域问答任务上的实验表明,EARS显著提升推测解码效率,在GSM8K基准上实现最高18.12%的吞吐量提升,准确率仅下降0.84%。该方法无需修改模型架构,可无缝集成至现有推测解码框架。

原文摘要 · Abstract (English)

Speculative Decoding is a prominent technique for accelerating the autoregressive inference of large language models (LLMs) by employing a fast draft model to propose candidate token sequences and a large target model to verify them in parallel. However, its core component -- the rejection sampling mechanism -- relies on a fixed, context-independent random threshold. This leads to a significant "random rejection" problem in high-uncertainty generation scenarios, where plausible candidate tokens are frequently rejected due to random chance, undermining inference efficiency. This paper introduces Efficient Adaptive Rejection Sampling (EARS), a novel method that dynamically adjusts the acceptance threshold by incorporating the target model's own predictive uncertainty, measured as 1 - max(P_target). By introducing a tolerance term proportional to this uncertainty, EARS intelligently relaxes the acceptance criterion when the model is uncertain, effectively reducing random rejections while maintaining strict standards when the model is confident. Experiments on creative writing and open-domain QA tasks demonstrate that EARS significantly enhances the efficiency of speculative decoding, achieving up to an 18.12% increase in throughput with a negligible 0.84% accuracy drop on the GSM8K benchmark. The method requires no modifications to model architectures and can be seamlessly integrated into existing speculative decoding frameworks.

推理加速拒绝采样大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。