用视频自动注入操作知识，让手机自动化更省力高效。
Mobile-Agent-V: A Video-Guided Approach for Effortless and Efficient Operational Knowledge Injection in Mobile Automation
- 用视频直接提取操作知识，无需手动编写脚本。
- 相比现有方法性能提升36%，大幅降低知识获取成本。
- 适合需要快速部署手机自动化流程的开发者和研究者。
移动设备使用量激增，亟需高效的自动化管理，但多数AI框架因缺乏实际操作经验而表现不佳。虽然人工编写知识可弥补这一缺陷，却耗时费力。本文提出Mobile-Agent-V框架，利用视频作为引导，直接从视频内容中提取操作知识，实现免手动、高效的知识注入。为严谨评估该方法，我们构建了Mobile-Knowledge基准，用于衡量外部知识对移动智能体性能的影响。实验结果表明，Mobile-Agent-V相比现有方法性能提升36%，充分证明其在移动自动化中具备显著的高效性与便捷性。
原文摘要 · Abstract (English)
The exponential rise in mobile device usage necessitates streamlined automation for effective task management, yet many AI frameworks fall short due to inadequate operational expertise. While manually written knowledge can bridge this gap, it is often burdensome and inefficient. We introduce Mobile-Agent-V, an innovative framework that utilizes video as a guiding tool to effortlessly and efficiently inject operational knowledge into mobile automation processes. By deriving knowledge directly from video content, Mobile-Agent-V eliminates manual intervention, significantly reducing the effort and time required for knowledge acquisition. To rigorously evaluate this approach, we propose Mobile-Knowledge, a benchmark tailored to assess the impact of external knowledge on mobile agent performance. Our experimental findings demonstrate that Mobile-Agent-V enhances performance by 36% compared to existing methods, underscoring its effortless and efficient advantages in mobile automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。