用自动化框架生成可执行环境,提升机械臂类智能体的训练效果。
EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent

- 构建沙箱隔离的工作空间与连贯任务轨迹生成器,实现环境自动生成。
- 合成139个交互环境、约2万复杂任务,使模型性能提升最高11.9%。
- 适合研究自主智能体训练、强化学习环境构造的学者使用。
大模型范式已从被动语言接口转向能在状态化工作空间中执行长周期任务的自主机械臂类智能体。尽管代理强化学习(Agentic RL)为优化此类智能体提供了前景,但其扩展严重受限于交互式训练环境的极度稀缺。现有合成环境仅限于工具调用接口,难以满足机械臂类智能体端到端的真实需求。为此,我们提出EnvCraft,一个自动合成可执行环境与可扩展训练数据的框架。具体而言,EnvCraft采用环境合成引擎构建沙箱隔离的工作空间,并结合拓扑感知的数据生成引擎生成连贯的任务轨迹。总体上,我们合成了139个交互环境,包含约2万项复杂任务,用于Agentic RL训练。在Qwen3/3.5模型(8B-32B)上的实验表明,该方法在机械臂类基准上性能提升最高达+11.9%,在通用工具使用基准上提升+8.0%,同时推理令牌成本下降。结果证实,合成的可执行环境能为训练提供稳健且泛化的学习信号。
原文摘要 · Abstract (English)
The paradigm of LLMs has rapidly shifted from passive language interfaces to autonomous Claw-like agents that execute long-horizon tasks across stateful workspaces. While Agentic Reinforcement Learning (Agentic RL) provides a promising path to optimize these agents, its scaling is heavily bottlenecked by the severe scarcity of interactive training environments. Existing synthetic environments are strictly limited to tool-calling endpoints, rendering them insufficient for accommodating the end-to-end real-world demands of claw-like agents. To bridge this gap, we introduce EnvCraft, an automated framework for synthesizing executable environments and scalable training data. Specifically, EnvCraft employs an environment synthesis engine to build sandbox-isolated workspaces, alongside a topology-aware data generation engine to produce coherent task trajectories. Overall, we synthesize 139 interactive environments comprising approximately 20K complex tasks for Agentic RL training. Experiments on Qwen3/3.5 models (8B-32B) show that our method yields gains of up to +11.9% on Claw-style benchmarks and +8.0% on general tool-use benchmarks, with concurrent reductions in inference token cost. The results confirm that synthesized executable environments provide robust and generalizable learning signals for training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。