让大模型按需思考,快慢自适应,推理更高效。
SABER: Switchable and Balanced Training for Efficient LLM Reasoning
- 用强化学习训练模型,按预算分配思考长度
- 快思模式减少65.4%推理词数,准确率还提升3.6%
- 支持四种推理模式,适合不同场景的效率需求
大型语言模型(LLMs)通过思维链推理在复杂任务上表现优异,但统一应用导致推理成本过高。本文提出SABER(可切换且平衡的训练框架),赋予模型用户可控、基于令牌预算的推理能力。SABER首先分析每个训练样本的基础模型思考词数,并分配至预设预算层级。微调阶段,模型受系统提示和长度感知奖励引导,遵守分配预算。同时引入无思考示例,确保模型在关闭显式推理时仍可靠。SABER支持四种离散推理模式:NoThink、FastThink、CoreThink 和 DeepThink,实现延迟与推理深度间的灵活权衡。在数学推理(MATH、GSM8K)、代码生成(MBPP)和逻辑推理(LiveBench-Reasoning)上的广泛评估表明,SABER在严格预算下保持高精度,具备平滑退化特性及跨尺度、跨领域泛化能力。特别地,SABER-FastThink 在 MATH 基准上将推理长度减少65.4%,准确率反而提升3.6%。
原文摘要 · Abstract (English)
Large language models (LLMs) empowered by chain-of-thought reasoning have achieved impressive accuracy on complex tasks but suffer from excessive inference costs and latency when applied uniformly to all problems. We propose SABER (Switchable and Balanced Training for Efficient LLM Reasoning), a reinforcement learning framework that endows LLMs with user-controllable, token-budgeted reasoning. SABER first profiles each training example's base-model thinking token usage and assigns it to one of the predefined budget tiers. During fine-tuning, the model is guided by system prompts and length-aware rewards to respect its assigned budget. In parallel, we incorporate no-think examples to ensure the model remains reliable even when explicit reasoning is turned off. SABER further supports four discrete inference modes - NoThink, FastThink, CoreThink, and DeepThink, enabling flexible trade-offs between latency and reasoning depth. Extensive evaluations on math reasoning (MATH, GSM8K), code generation (MBPP), and logical reasoning (LiveBench-Reasoning) demonstrate that SABER achieves high accuracy under tight budgets, graceful degradation, and effective cross-scale and cross-domain generalization. In particular, SABER-FastThink cuts reasoning length by 65.4% and yields a 3.6% accuracy gain compared with the base model on the MATH benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。