通过两种算法提升大模型推理时计算资源的利用效率,显著降低出错概率。
Provable Scaling Laws for the Test-Time Compute of Large Language Models
- 采用淘汰赛或联赛制生成并筛选多个候选答案,实现高效推理
- 错误率随计算量增加呈指数或幂律下降,理论可证明
- 仅需黑箱大模型,无需额外验证器,适合各类实际任务
我们提出两种简单、合理且实用的算法,能够为大型语言模型(LLMs)在测试阶段的计算资源使用提供可证明的缩放规律。第一种是两阶段淘汰式算法:给定输入问题后,先生成多个候选解,再通过淘汰赛方式聚合最终输出。假设大模型以非零概率生成正确解,并在对比正确与错误解时表现优于随机猜测,我们理论上证明该算法的失败概率随测试时计算量增长呈指数或幂律衰减。第二种是两阶段联赛式算法,每个候选解基于其对多个对手的平均胜率进行评估,而非单次失败即淘汰。在类似但更稳健的假设下,其失败概率同样随计算量增加指数下降。两种算法均只需黑箱大模型,无需验证器或奖励模型,实现极简部署,适用于不同任务。通过多种模型和数据集的大量实验,我们验证了理论预测,并展示了两种算法出色的缩放性能。
原文摘要 · Abstract (English)
We propose two simple, principled and practical algorithms that enjoy provable scaling laws for the test-time compute of large language models (LLMs). The first one is a two-stage knockout-style algorithm: given an input problem, it first generates multiple candidate solutions, and then aggregate them via a knockout tournament for the final output. Assuming that the LLM can generate a correct solution with non-zero probability and do better than a random guess in comparing a pair of correct and incorrect solutions, we prove theoretically that the failure probability of this algorithm decays to zero exponentially or by a power law (depending on the specific way of scaling) as its test-time compute grows. The second one is a two-stage league-style algorithm, where each candidate is evaluated by its average win rate against multiple opponents, rather than eliminated upon loss to a single opponent. Under analogous but more robust assumptions, we prove that its failure probability also decays to zero exponentially with more test-time compute. Both algorithms require a black-box LLM and nothing else (e.g., no verifier or reward model) for a minimalistic implementation, which makes them appealing for practical applications and easy to adapt for different tasks. Through extensive experiments with diverse models and datasets, we validate the proposed theories and demonstrate the outstanding scaling properties of both algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。