用验证器取代生成器，让手机自动化更稳更快
Advancing Mobile GUI Agents: A Verifier-Driven Approach to Practical Deployment
- 用大模型做动作验证而非直接生成，提升决策可靠性
- 在多个基准上成功率超现有方法，最高达49%
- 每步延迟仅4.3秒，速度比之前快6.1倍
我们提出V-Droid，一种面向移动端GUI任务自动化的智能代理。与以往依赖大语言模型（LLM）直接生成动作的移动代理不同，V-Droid将LLM用作验证器，在最终决策前评估候选动作。为实现这一新范式，我们构建了完整框架：离散化动作空间结合仅预填充工作流以加速验证；成对进度偏好训练显著提升验证器决策能力；可扩展的人机协同标注方案高效获取大规模数据。V-Droid在多个公开移动端任务自动化基准上表现优异：在AndroidWorld上成功率达59.5%，在AndroidLab上为38.3%，在MobileAgentBench上为49%，分别超越现有代理5.2%、2.1%和9%。此外，每步延迟低至4.3秒，较现有代理快6.1倍。代码已开源：https://github.com/V-Droid-Agent/V-Droid。
原文摘要 · Abstract (English)
We propose V-Droid, a mobile GUI task automation agent. Unlike previous mobile agents that utilize Large Language Models (LLMs) as generators to directly generate actions at each step, V-Droid employs LLMs as verifiers to evaluate candidate actions before making final decisions. To realize this novel paradigm, we introduce a comprehensive framework for constructing verifier-driven mobile agents: the discretized action space construction coupled with the prefilling-only workflow to accelerate the verification process, the pair-wise progress preference training to significantly enhance the verifier's decision-making capabilities, and the scalable human-agent joint annotation scheme to efficiently collect the necessary data at scale. V-Droid obtains a substantial task success rate across several public mobile task automation benchmarks: 59.5% on AndroidWorld, 38.3% on AndroidLab, and 49% on MobileAgentBench, surpassing existing agents by 5.2%, 2.1%, and 9%, respectively. Furthermore, V-Droid achieves a remarkably low latency of 4.3s per step, which is 6.1x faster compared with existing mobile agents. The source code is available at https://github.com/V-Droid-Agent/V-Droid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。