根据题目难易动态分配计算资源，提升大模型测试效率
Strategic Scaling of Test-Time Compute: A Bandit Learning Approach
- 将测试时计算分配建模为赌博机学习问题，实时评估题目难度
- 在MATH-500等数据集上最高提升15.29%性能，显著优于均匀分配
- 适合需要高效推理的数学、编程类任务，尤其对难题优化明显
扩大测试时计算量已成为提升大语言模型性能的有效策略。然而，现有方法通常对所有查询均匀分配计算资源，忽略了查询难度的差异。为此，我们首次将测试时计算分配建模为新型赌博机学习问题，提出自适应算法，可实时估计查询难度并相应分配计算资源。相比均匀分配，我们的算法在较难查询上增加计算投入，同时保持简单查询的准确率；对于复杂查询，进一步学习优先处理可解实例，有效避免对不可解问题的过度计算。理论上证明了算法比均匀分配更具计算效率，并在数学与代码基准上实证验证了其有效性。具体而言，在MATH-500数据集上性能最高提升11.10%（相对提升15.04%），AIME25数据集上提升10.82%（相对14.44%），LiveCodeBench数据集上提升11.23%（相对15.29%）。
原文摘要 · Abstract (English)
Scaling test-time compute has emerged as an effective strategy for improving the performance of large language models. However, existing methods typically allocate compute uniformly across all queries, overlooking variation in query difficulty. To address this inefficiency, we formulate test-time compute allocation as a novel bandit learning problem and propose adaptive algorithms that estimate query difficulty on the fly and allocate compute accordingly. Compared to uniform allocation, our algorithms allocate more compute to challenging queries while maintaining accuracy on easier ones. Among challenging queries, our algorithms further learn to prioritize solvable instances, effectively reducing excessive computing on unsolvable queries. We theoretically prove that our algorithms achieve better compute efficiency than uniform allocation and empirically validate their effectiveness on math and code benchmarks. Specifically, our algorithms achieve up to an 11.10% performance improvement (15.04% relative) on the MATH-500 dataset, up to 10.82% (14.44% relative) on the AIME25 dataset, and up to an 11.23% performance improvement (15.29% relative) on the LiveCodeBench dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。