arXiv:2506.20332cs.AI2025-06ACL被引 14

让手机智能体学会边做边纠错,提升复杂操作能力。

Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training

  • 分三阶段训练:先对齐思维结构,再验证动作执行,最后多轮任务实战
  • 在中文手机应用上构建24,521条标注数据集,支持真实环境测试
  • 开源全部资源,适合做移动智能体研究的开发者使用

基于视觉语言模型的移动智能体已具备理解复杂指令与手机截图的能力,得益于如群体相对策略优化(GRPO)等强化学习范式。然而,现有方法多依赖离线训练或局部动作奖励,易陷入局部最优,难以有效探索与错误修正。我们发现,直接使用任务级奖励因GUI交互稀疏性常导致收敛困难。为此,提出系统化训练方案Mobile-R1,通过三级层次化课程实现原子动作执行与策略任务完成之间的衔接:(1) 格式对齐以规范推理结构,(2) 基于可验证动作反馈的在线探索,建立基础执行能力,(3) 多轮任务级训练结合真实环境,释放探索潜力与自我修正能力。该策略显著提升了智能体的探索与自纠能力(“顿悟”时刻)。此外,针对非英文生态中多样化GUI数据稀缺问题,构建覆盖28个应用、含24,521条高质量人工标注的中文移动端数据集,并设立包含500条轨迹的严格基准。所有资源(数据集、基准、模型权重、代码)将开源:https://mobile-r1.github.io/Mobile-R1/

原文摘要 · Abstract (English)

Vision-language model-based mobile agents have gained the ability to understand complex instructions and mobile screenshots, benefiting from reinforcement learning paradigms like Group Relative Policy Optimization (GRPO). However, existing approaches centers on offline training or local action-level rewards often trap agents in local optima, hindering effective exploration and error correction with the environment. Crucially, we find that directly applying task-level rewards often leads to convergence difficulties due to the sparse nature of GUI interactions. To address these challenges, we present \textbf{Mobile-R1}, a systematic training recipe that bridges atomic action execution and strategic task completion. We propose a hierarchical curriculum consisting of three stages: (1) format alignment for reasoning structure, (2) on-policy exploration with verifiable action feedback to ground basic execution, and (3) multi-turn task-level training with realistic environment to unlock exploration and self-correction. This hierarchical strategy effectively bootstraps the agent, significantly enhancing its capability for exploration and self-correction (the ``Eureka'' moments). Furthermore, addressing the critical scarcity of diverse GUI data in non-English ecosystems, we contribute a comprehensive Chinese mobile dataset covering 28 applications with 24,521 high-quality manual annotations, and establish a rigorous benchmark with 500 trajectories. We will open source all resources, including the dataset, benchmark, model weight, and codes: https://mobile-r1.github.io/Mobile-R1/.

移动智能体强化学习中文数据集自纠错

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。