对比图形和命令行界面的执行瓶颈，发现界面差异影响任务完成率。
GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents
- 设计相同任务与验证条件，仅限制交互方式，公平比较GUI与CLI表现。
- 最强GUI agent成功率达59.1%，原生技能CLI仅48.2%，但增强后达69.3%。
- GUI瓶颈在长流程可靠交互，CLI瓶颈在技能覆盖范围与可扩展性。
计算机使用代理可通过图形界面或程序化命令界面执行软件任务，但现有评估混淆了交互模态与任务、初始状态、验证器及允许动作的差异。我们引入一个包含440个桌面任务的匹配执行层基准，覆盖18个应用和12类工作流，其中屏幕仅有的GUI代理与技能驱动的CLI代理接收相同的任务目标、初始状态和最终状态验证器，且仅能执行本模态的原生操作。在此控制环境下，最强GUI代理达到59.1%的全通过率，优于最强原始技能CLI代理的48.2%；然而，验证器引导的技能增强使CLI成功率提升至69.3%，表明CLI性能差距主要源于技能覆盖不全而非模型能力本身。结果表明，GUI与CLI暴露不同的执行瓶颈：GUI代理受限于长时程工作流中可靠的具身交互，而CLI代理受限于其技能接口的覆盖范围与可扩展性。
原文摘要 · Abstract (English)
Computer-use agents can execute software tasks through either graphical interfaces or programmatic command interfaces, but existing evaluations confound interaction modality with differences in tasks, initial states, verifiers, and permitted actions. We introduce a matched execution-layer benchmark of 440 desktop tasks across 18 applications and 12 workflow categories, where screen-only GUI agents and skill-mediated CLI agents receive identical goals, states, and final-state verifiers while being restricted to modality-native actions. In this controlled setting, the strongest GUI agent reaches a 59.1% full pass rate, outperforming the strongest original-skill CLI agent at 48.2%; however, verifier-guided skill augmentation raises CLI success to 69.3%, showing that much of the CLI deficit comes from incomplete skill coverage rather than model capability alone. These results suggest that GUI and CLI expose different execution bottlenecks: GUI agents are limited by reliable grounded interaction over long-horizon workflows, whereas CLI agents are limited by the coverage and scalability of their skill interfaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。