arXiv:2603.07432cs.CVcs.CL2026-03被引 4

提出新基准与训练系统,提升移动端AI代理的泛化能力

Generalization in Online Reinforcement Learning for Mobile Agents

  • 将任务建模为上下文马尔可夫决策过程,设计三阶段泛化测试基准
  • 70亿参数模型在未见任务上提升26.1%,但跨模板和应用泛化仍受限
  • 开源完整训练系统,支持复现与公平对比,适合强化学习与移动智能研究者

基于GUI的移动代理通过理解自然语言指令并操作屏幕来自动化手机任务。尽管近期方法采用强化学习(RL)训练视觉-语言模型(VLM)代理,但因缺乏标准化基准与开源系统,泛化能力仍未被充分探索。本文将问题形式化为上下文马尔可夫决策过程(CMDP),提出 extbf{AndroidWorld-Generalization}基准,包含三个逐步挑战的泛化场景:未见任务实例、任务模板和应用。我们进一步构建集成分组相对策略优化(GRPO)与可扩展采样系统的强化学习训练框架,包含容器化基础设施、异步执行及错误恢复机制,实现可靠高效的训练。在该基准上的实验表明,强化学习使70亿参数的VLM代理在未见任务实例上超越监督微调基线,提升26.1%;但在未见模板上仅提升15.7%,未见应用上仅8.3%,凸显泛化挑战。初步实验显示,测试时少样本适应可改善未见应用表现,为未来研究提供方向。为促进可复现性与公平比较,我们开源完整训练系统,包括环境、任务套件、模型、提示配置及底层基础设施。

原文摘要 · Abstract (English)

Graphical user interface (GUI)-based mobile agents automate digital tasks on mobile devices by interpreting natural-language instructions and interacting with the screen. While recent methods apply reinforcement learning (RL) to train vision-language-model(VLM) agents in interactive environments with a primary focus on performance, generalization remains underexplored due to the lack of standardized benchmarks and open-source RL systems. In this work, we formalize the problem as a Contextual Markov Decision Process (CMDP) and introduce \textbf{AndroidWorld-Generalization}, a benchmark with three increasingly challenging regimes for evaluating zero-shot generalization to unseen task instances, templates, and applications. We further propose an RL training system that integrates Group Relative Policy Optimization (GRPO) with a scalable rollout collection system, consisting of containerized infrastructure and asynchronous execution % , and error recovery to support reliable and efficient training. Experiments on AndroidWorld-Generalization show that RL enables a 7B-parameter VLM agent to surpass supervised fine-tuning baselines, yielding a 26.1\% improvement on unseen instances but only limited gains on unseen templates (15.7\%) and apps (8.3\%), underscoring the challenges of generalization. As a preliminary step, we demonstrate that few-shot adaptation at test-time improves performance on unseen apps, motivating future research in this direction. To support reproducibility and fair comparison, we open-source the full RL training system, including the environment, task suite, models, prompt configurations, and the underlying infrastructure \footnote{https://github.com/zihuanjiang/AndroidWorld-Generalization}.

强化学习移动代理泛化能力VLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。