让大模型推理更灵活可控,资源紧张时也不掉链子。
Scalable Chain of Thoughts via Elastic Reasoning
- 分两阶段推理:先思考后解题,各自分配资源预算
- 在严格限制下仍保持高可靠性,比基线少耗训练成本
- 适合需要低延迟、低算力的部署场景,如移动端
大型推理模型(LRMs)通过生成长链条思维(CoT)在复杂任务上取得显著进展。然而,其输出长度不可控,给实际部署带来挑战,因推理时的令牌数、延迟或计算资源受限。我们提出弹性推理(Elastic Reasoning),一种可扩展的链条思维框架,将推理明确分为独立分配预算的‘思考’与‘求解’两个阶段。测试时,该方法优先保证求解部分完整性,显著提升在严格资源约束下的可靠性。为训练对截断思考具有鲁棒性的模型,我们引入轻量级预算约束的滚动策略,集成于GRPO中,使模型学会在思考过程被中断时自适应推理,并在未见过的预算约束下有效泛化,无需额外训练。在数学(AIME、MATH500)和编程(LiveCodeBench、Codeforces)基准上的实证结果表明,弹性推理在严格预算约束下表现稳健,且训练成本远低于基线方法。尤为突出的是,即使在无约束环境下,该方法也生成更简洁高效的推理过程。代码已开源:https://github.com/SalesforceAIResearch/Elastic-Reasoning。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) have achieved remarkable progress on complex tasks by generating extended chains of thought (CoT). However, their uncontrolled output lengths pose significant challenges for real-world deployment, where inference-time budgets on tokens, latency, or compute are strictly constrained. We propose Elastic Reasoning, a novel framework for scalable chain of thoughts that explicitly separates reasoning into two phases--thinking and solution--with independently allocated budgets. At test time, Elastic Reasoning prioritizes the completeness of solution segments, significantly improving reliability under tight resource constraints. To train models that are robust to truncated thinking, we introduce a lightweight budget-constrained rollout strategy, integrated into GRPO, which teaches the model to reason adaptively when the thinking process is cut short and generalizes effectively to unseen budget constraints without additional training. Empirical results on mathematical (AIME, MATH500) and programming (LiveCodeBench, Codeforces) benchmarks demonstrate that Elastic Reasoning performs robustly under strict budget constraints, while incurring significantly lower training cost than baseline methods. Remarkably, our approach also produces more concise and efficient reasoning even in unconstrained settings. Our code has been made available at https://github.com/SalesforceAIResearch/Elastic-Reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。