将任意软件快速转为可交互的智能体环境,支持超长任务训练与验证。
Gym-Anything: Turn any Software into an Agent Environment
- 用多智能体协作自动配置软件环境,实现规模化部署。
- 构建包含超1万条长序列任务的CUA-World基准,最长任务超500步。
- 支持真实场景应用,适合研究复杂数字任务的智能体系统。
计算机使用智能体有望协助各类数字经济活动。但当前研究多集中于有限软件上的短时任务,经济价值有限,主要因环境构建耗时费力且难扩展。为此,我们提出Gym-Anything框架,将任意软件转化为可交互的计算机使用环境。环境创建被建模为多智能体任务:编码智能体编写脚本、下载真实数据并配置软件,生成设置证据;独立审计智能体则基于质量清单验证配置正确性。基于美国GDP数据构建的职业分类体系,我们应用该流程于200款软件,覆盖广泛职业领域。结果形成CUA-World,包含超过10,000条长时序任务,涵盖医学、天文学、工程及企业系统等,每项任务均配备真实数据及训练/测试划分。其中,CUA-World-Long是挑战性长时序基准,任务常需超过500步,远超现有水平。从训练集成功轨迹中蒸馏出的20亿参数视觉语言模型(VLM),性能优于自身规模两倍的模型。我们还在测试阶段引入审计机制:独立VLM审查完成轨迹并提供反馈,使Gemini-3-Flash在CUA-World-Long上的表现从11.5%提升至14.0%。所有代码、基础设施与基准数据已公开,推动真实世界计算机使用智能体研究。
原文摘要 · Abstract (English)
Computer-use agents hold the promise of assisting in a wide range of digital economic activities. However, current research has largely focused on short-horizon tasks over a limited set of software with limited economic value, such as basic e-commerce and OS-configuration tasks. A key reason is that creating environments for complex software requires significant time and human effort, and therefore does not scale. To address this, we introduce Gym-Anything, a framework for converting any software into an interactive computer-use environment. We frame environment creation itself as a multi-agent task: a coding agent writes setup scripts, downloads real-world data, and configures the software, while producing evidence of correct setup. An independent audit agent then verifies evidence for the environment setup against a quality checklist. Using a taxonomy of economically valuable occupations grounded in U.S. GDP data, we apply this pipeline to 200 software applications with broad occupational coverage. The result is CUA-World, a collection of over 10K long-horizon tasks spanning domains from medical science and astronomy to engineering and enterprise systems, each configured with realistic data along with train and test splits. CUA-World also includes CUA-World-Long, a challenging long-horizon benchmark with tasks often requiring over 500 steps, far exceeding existing benchmarks. Distilling successful trajectories from the training split into a 2B vision-language model outperforms models 2$\times$ its size. We also apply the same auditing principle at test time: a separate VLM reviews completed trajectories and provides feedback on what remains, improving Gemini-3-Flash on CUA-World-Long from 11.5% to 14.0%. We release all code, infrastructure, and benchmark data to facilitate future research in realistic computer-use agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。