让大模型自动调度推理方式,提升效率与准确率
Learning Adaptive Parallel Reasoning with Language Models
- 动态切换串行与并行推理,实现端到端自适应
- 4千上下文时准确率83.4%,比传统方法高23.4个百分点
- 适合需要高效推理的复杂任务场景
增大推理阶段计算量显著提升了语言模型的推理能力。然而,现有方法存在明显局限:串行思维链生成过长输出,导致延迟增加且超出上下文窗口;并行方法如自一致性缺乏有效协调,造成重复计算且性能提升有限。为此,我们提出自适应并行推理(APR),一种新型推理框架,使语言模型能够端到端地协调串行与并行计算。APR通过引入spawn()和join()操作,实现多线程推理的自适应调度。其核心创新在于端到端强化学习策略,优化父线程与子线程以提高任务成功率,无需预设推理结构。在Countdown推理任务上的实验表明:(1) 相同上下文窗口下性能更优(4k上下文时准确率达83.4%,对比60.0%);(2) 计算量增加时更具可扩展性(20k总词元时达80.1%,对比66.6%);(3) 相当延迟下精度更高(约5000毫秒时达75.2%,对比57.3%)。APR标志着语言模型向自主优化推理过程迈出重要一步。
原文摘要 · Abstract (English)
Scaling inference-time computation has substantially improved the reasoning capabilities of language models. However, existing methods have significant limitations: serialized chain-of-thought approaches generate overly long outputs, leading to increased latency and exhausted context windows, while parallel methods such as self-consistency suffer from insufficient coordination, resulting in redundant computations and limited performance gains. To address these shortcomings, we propose Adaptive Parallel Reasoning (APR), a novel reasoning framework that enables language models to orchestrate both serialized and parallel computations end-to-end. APR generalizes existing reasoning methods by enabling adaptive multi-threaded inference using spawn() and join() operations. A key innovation is our end-to-end reinforcement learning strategy, optimizing both parent and child inference threads to enhance task success rate without requiring predefined reasoning structures. Experiments on the Countdown reasoning task demonstrate significant benefits of APR: (1) higher performance within the same context window (83.4% vs. 60.0% at 4k context); (2) superior scalability with increased computation (80.1% vs. 66.6% at 20k total tokens); (3) improved accuracy at equivalent latency (75.2% vs. 57.3% at approximately 5,000ms). APR represents a step towards enabling language models to autonomously optimize their reasoning processes through adaptive allocation of computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。