用小模型快速生成候选答案，提升大模型推理速度与效率
$\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts
- 用小模型生成候选序列，大模型与奖励模型联合评估
- 在MATH500等数据集上准确率超束搜索，延迟降低19.1%
- 适合对响应速度敏感的推理场景，如实时问答系统
测试时扩展计算已推动大语言模型推理能力的提升，通常通过增加计算资源实现更充分的探索。然而，计算量增加往往带来更高的用户端延迟，直接影响体验。现有方法主要基于总算力（FLOPS）优化准确率，常忽略延迟约束。为此，我们提出SPECS，一种受推测解码启发的延迟感知测试时扩展方法。SPECS利用更小、更快的模型高效生成候选序列，并结合大模型与专用奖励模型的信号进行评估。引入新的集成策略，包括奖励引导的软验证和基于奖励的延后机制。在MATH500、AMC23和OlympiadBench数据集上的实验表明，SPECS在匹配或超越束搜索准确率的同时，延迟降低高达约19.1%。理论分析显示，随着束宽增加，算法收敛至带KL正则化的强化学习目标解。
原文摘要 · Abstract (English)
Scaling test-time compute has driven the recent advances in the reasoning capabilities of large language models (LLMs), typically by allocating additional computation for more thorough exploration. However, increased compute often comes at the expense of higher user-facing latency, directly impacting user experience. Current test-time scaling methods primarily optimize for accuracy based on total compute resources (FLOPS), often overlooking latency constraints. To address this gap, we propose $\texttt{SPECS}$, a latency-aware test-time scaling method inspired by speculative decoding. $\texttt{SPECS}$~uses a smaller, faster model to generate candidate sequences efficiently, and evaluates these candidates using signals from both a larger target model and a dedicated reward model. We introduce new integration strategies, including reward-guided soft verification and a reward-based deferral mechanism. Empirical results on MATH500, AMC23 and OlympiadBench datasets show that $\texttt{SPECS}$~matches or surpasses beam search accuracy while reducing latency by up to $\sim$19.1\%. Our theoretical analysis shows that our algorithm converges to the solution of a KL-regularized reinforcement learning objective with increasing beam width.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。