用小模型辅助大模型，提速降耗地提升推理准确率。
Guided Speculative Inference for Efficient Test-Time Alignment of LLMs
- 用小模型生成候选输出，结合奖励模型筛选最优解
- 在多个数学和科学任务上准确率超越现有方法，延迟降低28%
- 适合需要高效高精度推理的AI系统开发者
我们提出一种名为引导式推测推理（Guided Speculative Inference, GSI）的新算法，用于大型语言模型中高效的奖励引导解码。GSI将软版最佳n选一（soft best-of-$n$）测试时扩展与奖励模型 $r(x,y)$ 结合，并利用小辅助模型 $π_S(yackslash x)$ 生成推测样本。理论上，GSI可逼近基于基础模型 $π_B$ 的软版最佳n选一最优倾斜策略 $π_{β,B}(yackslash x) \propto π_B(yackslash x)\ ext{exp}(β\,r(x,y))$ 及最优策略下的期望奖励。在推理基准测试（MATH500、OlympiadBench、Minerva Math、MMLU-STEM、GSM8K）上，不同模型家族下，该方法在保持更高准确率的同时，相比标准软版最佳n选一（使用 $π_S$）及奖励引导推测解码（Liao et al., 2025），端到端延迟最多减少28%；在某些场景下甚至优于使用 $π_B$ 的软版最佳n选一。代码已开源：https://github.com/j-geuter/GSI。
原文摘要 · Abstract (English)
We propose Guided Speculative Inference (GSI), a novel algorithm for efficient reward-guided decoding in large language models. GSI combines soft best-of-$n$ test-time scaling with a reward model $r(x,y)$ and speculative samples from a small auxiliary model $π_S(y\mid x)$. We provably approximate both the optimal tilted policy $π_{β,B}(y\mid x) \propto π_B(y\mid x)\exp(β\,r(x,y))$ of soft best-of-$n$ under the base model $π_B$, as well as the expected reward under the optimal policy. In experiments on reasoning benchmarks (MATH500, OlympiadBench, Minerva Math, MMLU-STEM, GSM8K) and across different model families, our method achieves higher accuracy than standard soft best-of-$n$ with $π_S$ and reward-guided speculative decoding (Liao et al., 2025), and in certain settings even outperforms soft best-of-$n$ with $π_B$, while reducing end-to-end latency by up to $28\%$. The code is available at https://github.com/j-geuter/GSI .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。