在数据稀缺下构建安卓语言代理，无需人工标注轨迹
AndroidGen: Building an Android Language Agent under Data Scarcity
- 用自动生成轨迹替代人工标注，解决数据短缺问题
- 在多个应用上验证，显著提升LLM在安卓上的任务完成率
- 开源框架、模型与数据，适合移动智能体研究者使用
大型语言模型为多种自然语言处理任务带来了新可能，但其作为真实移动设备上的智能体尚未普及。主要瓶颈在于高质量数据的缺乏：人工标注耗时费力，且现有LLM存在完成率不足的问题。为此，我们提出AndroidGen框架，在数据稀缺条件下增强基于LLM的安卓代理能力。通过AndroidGen生成人类任务对应的执行轨迹，并利用这些轨迹训练开源LLM，构建无需人工标注轨迹的开源移动端智能体。我们在AndroidWorld、AitW及多个主流应用上进行了广泛评估，验证了该方法的有效性，同时揭示了未来改进方向。代码、模型与数据已公开于https://github.com/THUDM/AndroidGen。
原文摘要 · Abstract (English)
Large language models have opened up a world of possibilities for various NLP tasks, sparking optimism for the future. Despite their potential, LLMs have yet to be widely used as agents on real mobile devices. The main challenge is the need for high-quality data sources. Time constraints and labor intensity often hinder human annotation. On the other hand, existing LLMs exhibit inadequate completion rates and need a robust data filtration strategy. Given these challenges, we develop a framework called AndroidGen to enhance the capabilities of LLM-based agents under data scarcity. In addition, we leverage AndroidGen to collect trajectories given human tasks and train open-source LLMs on these trajectories to develop an open-source mobile agent without manually labeled trajectories. We extensively evaluate AndroidGen with AndroidWorld, AitW, and various popular applications, demonstrating its improvements and revealing potential areas for future improvement. Code, model, and data are available at https://github.com/THUDM/AndroidGen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。