让智能体学会自我分派任务，实现推理时的自动扩展。
Recursive Agent Optimization
- 训练智能体在推理时递归调用自身处理复杂任务
- 可突破模型上下文限制，解决比训练时更难的问题
- 适合需要长程推理和高效计算的AI系统开发者
我们提出递归智能体优化（RAO），一种用于训练递归智能体的强化学习方法：这类智能体可在运行时生成并递归地将子任务委派给自身的新实例。递归智能体实现了推理时的缩放算法，能通过分而治之自然扩展到更长的上下文，并泛化到更复杂的任务。RAO提供了一种训练方法，使模型学会何时以及如何委派与通信。实验表明，经过此方式训练的递归智能体具有更高的训练效率，可处理超出模型上下文窗口的任务，泛化能力远超训练任务难度，并相比单智能体系统显著降低实际运行时间。
原文摘要 · Abstract (English)
We introduce Recursive Agent Optimization (RAO), a reinforcement learning approach for training recursive agents: agents that can spawn and delegate sub-tasks to new instantiations of themselves recursively. Recursive agents implement an inference-time scaling algorithm that naturally allows agents to scale to longer contexts and generalize to more difficult problems via divide-and-conquer. RAO provides a method to train models to best take advantage of such recursive inference, teaching agents when and how to delegate and communicate. We find that recursive agents trained in this way enjoy better training efficiency, can scale to tasks that go beyond the model's context window, generalize to tasks much harder than the ones the agent was trained on, and can enjoy reduced wall-clock time compared to single-agent systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。