构建开放智能体学习生态,训练出能持续迭代的音乐创作智能体
Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem
- 基于三组件生态框架,实现智能体多轮交互与持续优化
- 在百万级轨迹上训练出性能优异的ROME模型,通过语义块信用分配提升长程稳定
- 适合研究智能体系统、具身学习或艺术生成的开发者参考
智能体创作要求大模型在真实环境中多轮执行动作、观察结果并迭代优化产物。尽管重要,开源社区仍缺乏系统化的端到端智能体开发生态。我们提出智能体学习生态系统(ALE),包含三个组件:ROLL(权重优化后训练框架)、ROCK(轨迹生成沙箱环境管理器)和iFlow CLI(高效上下文工程智能体框架)。我们发布基于ALE训练的开源智能体ROME,其训练数据来自超过一百万条轨迹。方法包括复杂行为合成的数据组合协议,以及新型策略优化算法——交互感知智能体策略优化(IPA),该算法基于语义交互块分配信用而非单个词元,显著提升长时程训练稳定性。我们在结构化环境下评估ROME,并引入终端基准测试Pro(Terminal Bench Pro),具备更强规模与污染控制能力。实验表明,ROME在SWE-bench Verified和Terminal Bench等基准上表现优异,验证了ALE的有效性。
原文摘要 · Abstract (English)
Agentic crafting requires LLMs to operate in real-world environments over multiple turns by taking actions, observing outcomes, and iteratively refining artifacts. Despite its importance, the open-source community lacks a principled, end-to-end ecosystem to streamline agent development. We introduce the Agentic Learning Ecosystem (ALE), a foundational infrastructure that optimizes the production pipeline for agentic model. ALE consists of three components: ROLL, a post-training framework for weight optimization; ROCK, a sandbox environment manager for trajectory generation; and iFlow CLI, an agent framework for efficient context engineering. We release ROME, an open-source agent grounded by ALE and trained on over one million trajectories. Our approach includes data composition protocols for synthesizing complex behaviors and a novel policy optimization algorithm, Interaction-Perceptive Agentic Policy Optimization (IPA), which assigns credit over semantic interaction chunks rather than individual tokens to improve long-horizon training stability. Empirically, we evaluate ROME within a structured setting and introduce Terminal Bench Pro, a benchmark with improved scale and contamination control. ROME demonstrates strong performance across benchmarks like SWE-bench Verified and Terminal Bench, proving the effectiveness of ALE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。