arXiv:2608.02352cs.LGcs.CL2026-08

Qwen-CUA让AI直接操控电脑,无需代码或特定接口,通用性强。

Qwen-CUA: Native Computer Use for (almost) Everything

论文配图:Qwen-CUA: Native Computer Use for (almost) Everything
图 1 · 摘自论文原文
  • 纯视觉+键盘鼠标操作,不依赖网页结构或专用API。
  • 在8个基准上超越Qwen3.7,OSWorld-Verified达86.2分。
  • 适合需要真实软件交互的自动化任务,如办公、测试等场景。

原生计算机使用为智能体提供通用接口,可操作几乎所有人类可用的软件,但需长时状态追踪、大规模交互经验及从稀疏但可验证结果中学习。我们提出Qwen-CUA,基于397B-A17B参数量的Qwen混合专家模型,仅通过截图观察并以键盘鼠标事件执行动作,不依赖DOM树、可访问性元数据或任务专用API。其架构维护最多20张活跃截图,并将旧视觉历史以固定大小块折叠,保留近期证据同时复用提示前缀。训练方面,构建包含近10万虚拟CPU和数万并发环境的云部署集群,构造约4万条可验证任务,收集日常与专业软件中的个性化长期工作流。通过可验证奖励与轨迹切片优化完整轨迹,迭代训练更新监督数据并重校强化学习任务。在八个基准测试中,Qwen-CUA优于Qwen3.7,OSWorld-Verified得分为86.2,OSWorld 2.0二进制/部分完成率分别为18.5/48.4。将同一方案扩展至超万亿参数模型后,Qwen-CUA-Max进一步提升至87.6与21.2/53.3。此外,其对红队攻击的防御能力也显著增强,成功率由36.6降至16.4。效率分析、浏览器部署及结合Bash的实验进一步刻画其实际行为。结果表明,原生计算机使用是通用智能体的重要基础,可验证交互与混合工具利用是关键发展方向。

原文摘要 · Abstract (English)

Native computer use offers a general interface for agents to operate almost any software available to people, but requires long-horizon state tracking, large-scale interactive experience, and learning from sparse yet verifiable outcomes. We introduce Qwen-CUA, a native computer-use agent with a 397B-A17B Qwen mixture-of-experts backbone. It observes only screenshots and acts through keyboard and mouse events, without DOM trees, accessibility metadata, or task-specific APIs. Its scaffold maintains up to 20 active screenshots and folds older visual history in fixed-size blocks to retain recent evidence while preserving reusable prompt prefixes. For training, we build a cloud rollout fleet with access to nearly 100,000 vCPUs and tens of thousands of concurrent environments, construct approximately 40,000 verifiable tasks, and collect personalized long-horizon workflows across everyday and professional software. We optimize complete trajectories with verifiable rewards and trajectory slicing, while iterative training runs refresh supervised data and recalibrate reinforcement-learning tasks. Across eight benchmarks, Qwen-CUA outperforms Qwen3.7 and remains competitive with leading proprietary systems, reaching 86.2 on OSWorld-Verified and 18.5/48.4 binary/partial completion on OSWorld 2.0. Scaling the same recipe to a model with over one trillion parameters yields Qwen-CUA-Max, improving these scores to 87.6 and 21.2/53.3. Qwen-CUA also reduces RedTeamCUA attack success from 36.6 to 16.4 relative to Qwen3.7. Efficiency analyses, a browser deployment, and Bash-augmented experiments further characterize practical behavior. These results establish native computer use as a broadly capable agent foundation and highlight scalable verifiable interaction and hybrid tool use as key directions.

智能体人机交互自动化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。