arXiv:2505.15810cs.CLcs.AI2025-05NeurIPS被引 70

针对GUI智能体视觉定位训练中的三大缺陷,提出三类优化方案,性能超越同类模型。

GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents

  • 采用快速思维模板减少冗余推理,提升定位效率
  • 引入框大小约束,防止奖励作弊,改善定位精度
  • 改进策略更新机制,增强对难样本的优化能力

近期图形用户界面(GUI)智能体借鉴R1-Zero范式,结合在线强化学习与显式思维链推理,在对象定位前实现显著性能提升。本文针对该训练流程中的三个关键组件展开深入分析:输入设计、输出评估与策略更新,发现盲目套用通用强化学习会引发特定问题。输入设计方面,现有模板虽促进思维链生成,但更长的推理链反而降低定位效果;输出评估中,基于命中信号或框面积的奖励函数导致模型通过扩大框尺寸作弊,影响定位质量;策略更新阶段,在线强化学习因长度与样本难度偏差而过拟合简单样本,忽视难点。为此,本文提出三项改进:一、采用快速思维模板,鼓励直接作答,减少训练中的过度推理;二、在奖励函数中加入框大小约束,抑制奖励作弊;三、调整长度归一化并引入难度感知缩放因子,优化困难样本的学习。基于Qwen2.5-VL-3B-Instruct,在17,000个公开样本上训练的GUI-G1-3B模型,在ScreenSpot上达到90.3%准确率,在ScreenSpot-Pro上达37.1%,优于同规模所有先前模型,甚至超越更大规模的UI-TARS-7B,创下新纪录。项目代码已开源。

原文摘要 · Abstract (English)

Recent Graphical User Interface (GUI) agents replicate the R1-Zero paradigm, coupling online Reinforcement Learning (RL) with explicit chain-of-thought reasoning prior to object grounding and thereby achieving substantial performance gains. In this paper, we first conduct extensive analysis experiments of three key components of that training pipeline: input design, output evaluation, and policy update-each revealing distinct challenges arising from blindly applying general-purpose RL without adapting to GUI grounding tasks. Input design: Current templates encourage the model to generate chain-of-thought reasoning, but longer chains unexpectedly lead to worse grounding performance. Output evaluation: Reward functions based on hit signals or box area allow models to exploit box size, leading to reward hacking and poor localization quality. Policy update: Online RL tends to overfit easy examples due to biases in length and sample difficulty, leading to under-optimization on harder cases. To address these issues, we propose three targeted solutions. First, we adopt a Fast Thinking Template that encourages direct answer generation, reducing excessive reasoning during training. Second, we incorporate a box size constraint into the reward function to mitigate reward hacking. Third, we revise the RL objective by adjusting length normalization and adding a difficulty-aware scaling factor, enabling better optimization on hard samples. Our GUI-G1-3B, trained on 17K public samples with Qwen2.5-VL-3B-Instruct, achieves 90.3% accuracy on ScreenSpot and 37.1% on ScreenSpot-Pro. This surpasses all prior models of similar size and even outperforms the larger UI-TARS-7B, establishing a new state-of-the-art in GUI agent grounding. The project repository is available at https://github.com/Yuqi-Zhou/GUI-G1.

GUI智能体视觉定位强化学习奖励设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。