让大模型重点攻克难题,高效分配算力提升数学推理能力
SCALE: Selective Resource Allocation for Overcoming Performance Bottlenecks in Mathematical Test-time Scaling
- 按题目难易度差异分配算力,难的用强模式,简单的快速处理
- 在AIME25上准确率从57.5%提升至71.25%,提升13.75个百分点
- 适合追求高精度且资源受限的数学推理应用
测试时计算扩展已成为增强大语言模型数学推理能力的有效范式,通过在推理阶段分配额外计算资源实现性能提升。然而,现有方法对所有推理子问题采用均匀资源分配,导致难题获得关注不足,而简单操作却消耗过多资源,造成性能瓶颈,额外算力带来的收益递减。受双过程理论启发,我们提出 extbf{SCALE}(Selective Resource Allocation)框架,根据子问题难度选择性分配计算资源。SCALE包含四个阶段:(1) 将问题分解为连续的推理子问题;(2) 评估每个子问题的难度,区分常规操作与计算挑战性任务;(3) 为简单子问题分配系统1模式,复杂子问题使用系统2模式;(4) 顺序执行并传递上下文。通过聚焦资源于难题,同时高效处理常规操作,SCALE实现了显著性能提升和更优资源利用率。大量实验表明,相比均匀扩展基线,SCALE显著优于后者,在AIME25上准确率提升最高达13.75个百分点(从57.50%增至71.25%),同时计算成本降低33%-53%,是测试时扩展领域的重要进展,解决了当前方法的根本局限。
原文摘要 · Abstract (English)
Test-time compute scaling has emerged as a powerful paradigm for enhancing mathematical reasoning in large language models (LLMs) by allocating additional computational resources during inference. However, current methods employ uniform resource distribution across all reasoning sub-problems, creating fundamental bottlenecks where challenging sub-problems receive insufficient attention while routine operations consume disproportionate resources. This uniform allocation creates performance bottlenecks where additional computational resources yield diminishing returns. Inspired by dual-process theory, we propose \textbf{SCALE} (Selective Resource Allocation), a framework that selectively allocates computational resources based on sub-problem difficulty. SCALE operates through four stages: (1) problem decomposition into sequential reasoning sub-problems, (2) difficulty assessment of each sub-problem to distinguish between routine operations and computationally challenging sub-problems, (3) selective processing mode assignment between System 1 for simple sub-problems and System 2 for complex ones, and (4) sequential execution with context propagation. By concentrating resources on challenging sub-problems while processing routine operations efficiently, SCALE achieves substantial performance improvements with superior resource utilization. Extensive experiments demonstrate that SCALE significantly outperforms uniform scaling baselines, achieving accuracy improvements of up to 13.75 percentage points (57.50% to 71.25% on AIME25) while reducing computational costs by 33%-53%, representing a major advance in test-time scaling that addresses fundamental limitations of current approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。