通过追踪界面实体的'染色流'，让机器人智能更精准地判断操作步骤对错。
StainFlow: Entity-Stain Tracking and Evidence Linking for Process Rewards in GUI Agents
- 用实体染色流动态划分任务阶段，避免人为设定里程碑的主观性。
- 根据关键操作前后实体状态变化，动态构建证据窗口，提升判断准确率。
- 适合需要长序列决策的GUI自动化任务，尤其在多路径执行场景下表现优。
强化学习（RL）在长时序、随机性的数字环境中的图形用户界面（GUI）智能体中展现出巨大潜力，但轨迹级的成功反馈过于稀疏，难以对中间探索步骤进行可靠的信用分配。为缓解此问题，近期研究引入了过程奖励模型（PRMs），通过全局里程碑验证或局部步骤评估提供更细粒度的训练反馈。然而，这些方法仍存在两类特定局限：全局里程碑分解具有主观性和单一性，难以适应真实GUI任务中的多种有效执行路径；固定局部判断窗口可能遗漏长距离关键证据，或因无关帧稀释决策信号。受网络流分析中染色追踪机制启发，本文提出StainFlow——一种面向GUI智能体的实体染色流过程奖励模型。为降低全局划分的主观性，提出全局实体染色追踪模块，提取视觉可验证的任务实体，并追踪其染色浓度与状态沿轨迹的演化，通过实体证据流的变化客观划分任务阶段。为提升局部验证精度，提出局部染色证据链接模块，以每个候选关键节点的触发实体为中心，基于其染色浓度和状态变化检索相关步骤，动态构建高密度证据窗口以验证真实关键节点。在AndroidWorld和OGRBench上的大量实验表明，StainFlow相对提升了在线RL成功率3.2%，轨迹完成判断准确率1.8%。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) has become a promising approach for improving GUI Agents in long-horizon, stochastic digital environments, but trajectory-level success feedback is too sparse to provide reliable credit assignment for intermediate exploration steps. To mitigate this issue, recent studies introduce Process Reward Models (PRMs), which provide finer-grained training feedback through global milestone verification or local step-level evaluation. However, these methods still suffer from two level-specific limitations: global milestone decomposition is subjective and singular, making it difficult to accommodate the multiple valid execution paths in real GUI tasks, while fixed local judging windows may miss long-range key evidence or dilute the decision signal with irrelevant frames. Inspired by stain-tracing mechanisms in network flow analysis, we propose StainFlow, an entity-stain-flow process reward model for GUI Agents. To reduce the subjectivity of global partitioning, we introduce the Global Entity Stain Tracking module, which extracts visually verifiable task entities and tracks how their stain concentrations and states evolve along the trajectory, allowing task phases to be objectively separated by changes in the entity evidence flow. To improve the accuracy of local verification, we introduce the Local Stain Evidence Linking module. Centered on the triggering entities of each candidate key node, it retrieves relevant steps based on their stain concentrations and state changes, and dynamically constructs high-density evidence windows for verifying true key nodes. Extensive experiments on AndroidWorld and OGRBench show that StainFlow relatively improves online RL success by 3.2% and trajectory completion judgment accuracy by 1.8%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。