让AI主动检查界面操作对不对，减少错误累积。
GUI-PRA: Process Reward Agent for GUI Tasks
- 用动态标准主动查找界面证据，而非简单比对图像。
- 在安卓和移动任务上准确率提升5%~6.5%，达54.74%。
- 适合做复杂界面自动化、需要精准判断的场景。
长序列GUI自动化因错误累积仍具挑战性。现有过程奖励模型（PRMs）依赖表面视觉对齐，缺乏领域特定验证逻辑，且仅进行被动单次视觉判断，易受视觉模糊干扰。本文提出GUI-PRA，将界面评估从被动打分转为主动探查：结合经验注入的准则生成，提炼出状态相关的通用验证规则；再通过准则引导的自回归感知，利用多粒度视觉工具定位并获取任务相关证据。在AndroidWorld和Mobile-MiniWoB++数据集上，GUI-PRA相比标准PRMs分别提升5.0%和6.5%性能，Qwen3-VL在AndroidWorld上达到54.74%成功率；在离线OS-Critic Bench上表现媲美全训练批判模型。
原文摘要 · Abstract (English)
Long-horizon GUI automation remains challenging due to error accumulation over extended interaction sequences. Process Reward Models (PRMs) provide dense step-level supervision for mitigating error accumulation, yet standard PRMs are poorly suited to GUI verification. Standard PRM judgments often rely on superficial visual alignment rather than functional correctness, reflecting an evaluative knowledge gap caused by missing domain-specific adjudication logic. Standard PRMs also perform passive, single-pass visual assessment, which creates Visual Ambiguity when reliable judgment requires actively locating, parsing, or inspecting task-relevant UI evidence. We introduce GUI-PRA, a Process Reward Agent that transforms GUI process evaluation from passive scoring into active investigation. GUI-PRA couples Experience-Injected Criterion Synthesis, which distills generalized verification principles into state-specific criteria, with Criterion-Guided Autoregressive Perception, which uses these criteria to navigate multi-granularity visual tools and gather grounded evidence. On AndroidWorld and Mobile-MiniWoB++, GUI-PRA achieves improvements of 5.0% and 6.5% over standard PRMs on the Qwen-VL series, with Qwen3-VL attaining 54.74% success rate on AndroidWorld. On the offline OS-Critic Bench, GUI-PRA demonstrates strong competitiveness against fully trained critic models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。