用课程学习提升GUI任务机器人表现,更聪明地分步训练。
CRAFT-GUI: Curriculum-Reinforced Agent For GUI Tasks
- 按任务难易程度分阶段训练,避免统一难度带来的学习瓶颈。
- 在公开数据集上提升5.6%,内部测试提升10.3%。
- 适合想提升自动化交互系统性能的研究者与工程师。
随着自主智能体在图形用户界面(GUI)环境中的理解与交互能力不断增强,自动化任务执行的新时代正在到来。近期研究证明强化学习(RL)能有效提升代理在动态交互式GUI环境中的表现。然而,现有方法存在两大局限:(1)将全部训练数据视为同一难度,忽视了不同GUI任务间的显著差异,阻碍了学习过程的自适应;(2)多数方法将任务特异性细节压缩为单一粗粒度奖励信号,导致策略更新效率低下。为此,我们提出CRAFT-GUI,一种基于组相对策略优化(GRPO)的课程学习框架,显式建模轨迹间难度差异。为实现更精细的策略优化,设计融合规则基信号与模型评估的奖励函数,提供更丰富、更细致的训练反馈。实验表明,本方法在公开基准Android Control上超越当前最优方法5.6%,在内部在线基准上提升10.3%。结果验证了强化学习与课程学习结合在GUI交互任务中的有效性。
原文摘要 · Abstract (English)
As autonomous agents become adept at understanding and interacting with graphical user interface (GUI) environments, a new era of automated task execution is emerging. Recent studies have demonstrated that Reinforcement Learning (RL) can effectively enhance agents' performance in dynamic interactive GUI environments. However, these methods face two key limitations: (1) they overlook the significant variation in difficulty across different GUI tasks by treating the entire training data as a uniform set, which hampers the agent's ability to adapt its learning process; and (2) most approaches collapse task-specific nuances into a single, coarse reward, leaving the agent with a uniform signal that yields inefficient policy updates. To address these limitations, we propose CRAFT-GUI, a curriculum learning framework based on Group Relative Policy Optimization (GRPO) that explicitly accounts for the varying difficulty across trajectories. To enable more fine-grained policy optimization, we design a reward function that combines simple rule-based signals with model-judged evaluation, providing richer and more nuanced feedback during training. Experimental results demonstrate that our method achieves significant improvements over previous state-of-the-art approaches, outperforming them by 5.6% on public benchmarks Android Control and 10.3% on our internal online benchmarks, respectively. These findings empirically validate the effectiveness of integrating reinforcement learning with curriculum learning in GUI interaction tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。