用真实与模拟环境结合训练智能手机代理，提升任务成功率。
PhoneBuddy: Training Open Models for Agentic Phone Use
- 融合真实与模拟界面环境，分阶段训练手机代理。
- 混合训练使真实手机任务成功率达45.33%，模拟环境达83.2%。
- 适合研究手机智能体、人机交互与自动化应用的开发者。
手机正成为通用智能体的重要执行界面，但训练开放模型实现可靠手机操作仍具挑战：部署时的真实设备环境反应慢、状态依赖强、有副作用且难以复位或验证；而可扩展的模拟环境仅近似真实行为。本文提出PhoneBuddy，一种针对智能手机使用的训练方案与开源模型，结合真实应用环境与模拟应用环境PhoneWorld——后者从真实图形界面使用结构重建可运行的模拟应用。PhoneBuddy首先在双环境中收集轨迹进行共享监督微调，随后对比真实环境强化学习（RL）与混合环境（双环境）强化学习。在涵盖150个任务的人工评估中，真实手机上的任务成功率从监督微调后的36.67%提升至真实环境RL的40.67%，再升至混合环境RL的45.33%。在AndroidWorld上，对应数据为60.3% → 77.2% → 83.2%。结果表明，模拟环境并非真实环境强化学习的替代品，而是可扩展、可复位、自动验证的互补训练资源。提升最显著于单个应用及小应用任务，而跨应用长流程任务仍是关键挑战。
原文摘要 · Abstract (English)
Phones are becoming an important execution surface for general-purpose agents, but training open models for reliable phone use remains difficult because the environment that matters at deployment, real devices running real apps, is slow, stateful, side-effectful, and hard to reset or verify, while scalable mock environments only approximate real behavior. We present PhoneBuddy, a training recipe and open-model line for agentic phone use that combines a real-app environment with a mock-app environment, PhoneWorld, which reconstructs runnable mock apps from real GUI usage structure. PhoneBuddy first builds a shared supervised fine-tuning stage from trajectories collected in both environments, then compares real-app RL against mixed RL across both environments. Across a 150-task human evaluation on real phones spanning apps, mini-apps, and cross-app workflows, task success rate improves from 36.67\% after supervised fine-tuning to 40.67\% after real-app RL and 45.33\% after mixed RL. On AndroidWorld, the same progression rises from 60.3\% to 77.2\% to 83.2\%. These results show that mock-app training is not a replacement for real-app RL, but a complementary source of scalable, resettable, and automatically checked interaction. The gains are strongest on app and mini-app tasks, while long-horizontal cross-app workflows remain an important open challenge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。