通过推理与执行协同，提升GUI智能体的奖励准确性。
ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaboration
- 用通用推理器+领域执行器主动探查界面状态
- 在3000+轨迹上使奖励准确率提升5.3%，F1提升19.4%
- 适合需高精度奖励的GUI自动化研究者
奖励对大语言模型的评估与训练至关重要。然而，现有基于规则或模型的奖励方法难以泛化至GUI智能体场景，因缺乏真实轨迹或应用数据库，且静态轨迹式LLM评判方法准确率有限。为此，我们提出ProRe，一种主动奖励系统，通过通用推理器与领域特定评估器（执行器）协作实现。推理器调度针对性的状态探测任务，评估器则主动与环境交互以获取额外观测，从而提升推理器对GUI智能体奖励的准确性与可验证性。在超过3000条轨迹上的实证结果表明，ProRe将奖励准确率和F1分数分别提升最高5.3%和19.4%。进一步将ProRe集成至前沿策略智能体后，成功率最高提升22.4%。代码已开源：https://github.com/V-Droid-Agent/ProRe。
原文摘要 · Abstract (English)
Reward is critical to the evaluation and training of large language models (LLMs). However, existing rule-based or model-based reward methods struggle to generalize to GUI agents, where access to ground-truth trajectories or application databases is often unavailable, and static trajectory-based LLM-as-a-Judge approaches suffer from limited accuracy. To address these challenges, we propose ProRe, a proactive reward system that leverages a general-purpose reasoner and domain-specific evaluator agents (actors). The reasoner schedules targeted state probing tasks, which the evaluator agents then execute by actively interacting with the environment to collect additional observations. This enables the reasoner to assign more accurate and verifiable rewards to GUI agents. Empirical results on over 3K trajectories demonstrate that ProRe improves reward accuracy and F1 score by up to 5.3\% and 19.4\%, respectively. Furthermore, integrating ProRe with state-of-the-art policy agents yields a success rate improvement of up to 22.4\%. The source code is available at https://github.com/V-Droid-Agent/ProRe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。