用GPT-4验证每一步操作，让AI更高效学会用电脑
STEVE: A Step Verification Pipeline for Computer-use Agent Training
- 用GPT-4逐步验证操作正确性，生成二值标签
- 7B模型在WinAgentArena上达到领先性能，成本更低
- 适合想低成本训练真实桌面场景智能体的研究者
开发能自主操控图形界面的AI代理仍是长期挑战。近期数据规模定律启发我们通过大规模指令集训练计算机使用代理，但行为克隆仍需海量高质量轨迹。为此，我们设计了STEVE——一种步骤验证流水线：首先构建大规模计算机使用指令集，并收集由次优代理产生的轨迹；利用GPT-4o根据动作前后的屏幕状态判断每一步操作的正确性，赋予二值标签；最后采用Kahneman-Tversky优化方法，从这些二值标签中优化代理。大量实验表明，该方法通过利用轨迹中的正负样本，显著优于监督微调。此外，STEVE使我们成功训练了一个7B视觉语言模型作为计算机使用代理，在具有挑战性的实时桌面环境WinAgentArena上表现领先，且效率高、成本低。代码与数据见：https://github.com/FanbinLu/STEVE。
原文摘要 · Abstract (English)
Developing AI agents to autonomously manipulate graphical user interfaces is a long challenging task. Recent advances in data scaling law inspire us to train computer-use agents with a scaled instruction set, yet using behavior cloning to train agents still requires immense high-quality trajectories. To meet the scalability need, we designed STEVE, a step verification pipeline for computer-use agent training. First, we establish a large instruction set for computer-use agents and collect trajectory data with some suboptimal agents. GPT-4o is used to verify the correctness of each step in the trajectories based on the screens before and after the action execution, assigning each step with a binary label. Last, we adopt the Kahneman and Tversky Optimization to optimize the agent from the binary stepwise labels. Extensive experiments manifest that our agent outperforms supervised finetuning by leveraging both positive and negative actions within a trajectory. Also, STEVE enables us to train a 7B vision-language model as a computer-use agent, achieving leading performance in the challenging live desktop environment WinAgentArena with great efficiency at a reduced cost. Code and data: https://github.com/FanbinLu/STEVE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。