用强化学习让大模型自主解题，准确率提升近一倍。
L0: Reinforcement Learning to Become General Agents
- 通过代码即动作的循环机制训练智能体
- 仅靠可验证奖励的强化学习使准确率翻倍
- 适合想低成本训练通用智能体的研究者
训练大型语言模型作为自主代理完成多轮、长时程任务仍面临可扩展性和训练效率的重大挑战。为此，我们提出L-Zero（L0），一个可扩展的端到端通用智能体训练流水线。其具备低成本、可扩展且沙箱化的并发代理工作池，降低了在复杂环境中应用强化学习的门槛。我们还引入了NB-Agent——L0中的智能体框架，采用‘代码即动作’的读取-求值-打印循环（REPL）方式运行。我们在事实性问答基准上评估了L0。实验表明，仅使用强化学习与可验证奖励（RLVR），基础模型即可发展出稳健的问题解决能力。在Qwen2.5-7B-Instruct模型上，SimpleQA准确率从30%提升至80%，HotpotQA从22%提升至41%。我们已开源整个L0系统，包括L0系列模型、NB-Agent、完整训练流水线及训练方案，详见https://github.com/cmriat/l0。
原文摘要 · Abstract (English)
Training large language models (LLMs) to act as autonomous agents for multi-turn, long-horizon tasks remains significant challenges in scalability and training efficiency. To address this, we introduce L-Zero (L0), a scalable, end-to-end training pipeline for general-purpose agents. Featuring a low-cost, extensible, and sandboxed concurrent agent worker pool, L0 lowers the barrier for applying reinforcement learning in complex environments. We also introduce NB-Agent, the agent scaffold within L0, which operates in a "code-as-action" fashion via a Read-Eval-Print-Loop (REPL). We evaluate L0 on factuality question-answering benchmarks. Our experiments demonstrate that a base model can develop robust problem-solving skills using solely Reinforcement Learning with Verifiable Rewards (RLVR). On the Qwen2.5-7B-Instruct model, our method boosts accuracy on SimpleQA from 30 % to 80 % and on HotpotQA from 22 % to 41 %. We have open-sourced the entire L0 system, including our L0 series models, the NB-Agent, a complete training pipeline, and the corresponding training recipes on (https://github.com/cmriat/l0).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。