用可演化环境训练计算机使用智能体,显著提升其实际表现。
Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale

- 构建可自进化状态环境,任务与验证器随模型同步优化
- 90亿参数模型准确率从36.5%升至67.1%,接近大模型水平
- 适合研究通用智能体、强化学习与自动化系统的人群
计算机使用智能体通过动作带来的变化进行学习,因此训练需要能交互、可破坏并重置的应用环境。最需训练的是登录保护且具有状态的应用,而合成环境正填补这一空白。现有流水线虽能批量生成,但瓶颈转向单个环境内部质量。我们发现三个关键属性决定效果:环境的行为深度、是否针对智能体真实失败点、是否随模型共同进化。本文提出Echoverse,将规范编译为具状态的应用,任务基于应用内数据库评分,并设计双读回循环:既用于修复环境、任务与验证器,也作为模型训练信号。在十二个此类环境中训练的90亿参数模型,在十四项评估中准确率从36.5%提升至67.1%,仅差于指导它的大型前沿模型约14个百分点。对比实验显示:浅层环境使真实站点准确率从80.0%降至75.0%,深层环境则分别提升至85.0%和65.0%;单一界面控件的跨渲染迁移成功扩展至未见控件族与开放网页;单一环境修复使模型性能从16.2%升至38.5%。同一环境亦可作强化学习场景,结合基线验证器与密集步级判别器,使保留测试集得分从58.8%升至68.0%。我们发布四个环境作为基准,包含应用、种子数据与基线评分器。代码:https://aka.ms/echoverse
原文摘要 · Abstract (English)
Computer-use agents learn from what their actions change, so training one needs applications it can act on, break and reset. The applications that matter most are login-gated and stateful, so synthetic environments stand in for them. Recent pipelines generate such environments in bulk, which moves the bottleneck from how many exist to what is inside each one. The returns, we find, come from three properties: how much behavioural depth an environment carries, whether it targets the interaction an agent actually fails, and whether it improves alongside the model. We present Echoverse, which compiles specifications into stateful applications whose tasks are graded against the application's own database, and a co-evolution loop that reads every graded rollout twice: as repairs to the environment, its tasks and its verifier, and as training signal for the model. Trained on twelve such environments, a 9B model improves from $36.5\%$ to $67.1\%$ across fourteen evaluation splits, within fourteen points of the much larger frontier model that taught it. We examine each property in turn. On the same domains, shallow environments push live-site accuracy below the base model ($80.0 \to 75.0$) while deep ones raise it ($80.0 \to 85.0$ and $48.0 \to 65.0$); drilling one interface control across many renderings transfers to held-out widget families and to the open web; and repairing a single environment lifts the model trained on it from $16.2\%$ to $38.5\%$. The same worlds serve as reinforcement-learning environments, where a reward combining the grounded verifier with a dense per-step judge raises held-out score from $58.8\%$ to $68.0\%$. We release four environments as a benchmark, with their applications, seed data and grounded graders. Code: https://aka.ms/echoverse
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。