用强化学习让智能体自动为每个问题生成定制化解决方案。
FlowReasoner: Reinforcing Query-Level Meta-Agents
- 通过外部执行反馈强化推理型元智能体，实现个性化多智能体系统生成。
- 在三个基准上比o1-mini高10.52%准确率，兼顾性能与效率。
- 适合需要自动代码生成与复杂任务分解的研究者与开发者。
本文提出一种名为FlowReasoner的查询级元智能体，用于自动化设计针对每个用户查询的多智能体系统。核心思想是通过外部执行反馈激励基于推理的元智能体。首先，通过蒸馏DeepSeek R1，赋予FlowReasoner生成多智能体系统的基础推理能力；随后，利用强化学习（RL）并结合外部执行反馈进一步优化。设计了一个多目标奖励函数，从性能、复杂度和效率三个维度指导训练。由此，FlowReasoner能够通过深思熟虑的推理，为每个用户查询生成个性化的多智能体系统。在工程与竞赛代码基准上的实验表明其优越性：在三个基准上显著超越o1-mini，准确率提升10.52%。代码已开源于https://github.com/sail-sg/FlowReasoner。
原文摘要 · Abstract (English)
This paper proposes a query-level meta-agent named FlowReasoner to automate the design of query-level multi-agent systems, i.e., one system per user query. Our core idea is to incentivize a reasoning-based meta-agent via external execution feedback. Concretely, by distilling DeepSeek R1, we first endow the basic reasoning ability regarding the generation of multi-agent systems to FlowReasoner. Then, we further enhance it via reinforcement learning (RL) with external execution feedback. A multi-purpose reward is designed to guide the RL training from aspects of performance, complexity, and efficiency. In this manner, FlowReasoner is enabled to generate a personalized multi-agent system for each user query via deliberative reasoning. Experiments on both engineering and competition code benchmarks demonstrate the superiority of FlowReasoner. Remarkably, it surpasses o1-mini by 10.52% accuracy across three benchmarks. The code is available at https://github.com/sail-sg/FlowReasoner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。