arXiv:2605.14311cs.LGcs.AI2026-05

提出非二值化界面评分模型,提升智能体操作判断精度。

Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment

论文配图:Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment
图 1 · 摘自论文原文
  • 用两阶段对比学习构建连续语义对齐的评分机制
  • 在四层分类体系下实现比7B模型更优的零样本跨平台表现
  • 适用于需细粒度动作评估的通用界面智能体研发

测试时扩展(TTS)通过采样多个候选操作并由评判模型排序,已成为通用图形用户界面(GUI)智能体的有前景范式。其有效性取决于评判模型的细粒度排序能力。然而现有GUI评判模型均采用二值分类,我们对其动机分析揭示了严重纠缠问题:有效动作与看似合理但无效的干扰项得分无法区分。这一失败归因于两个结构缺陷:可操作性坍塌——层次化可操作空间被压缩为0/1标签;噪声敏感性——二值目标过拟合于噪声决策边界。为此,我们提出BBCritic(非二值评判器),基于功能等价假设,通过两阶段对比学习,在共享可操作空间中对齐指令与动作,恢复二值监督所压平的层次结构。同时提出BBBench(非二值基准),首个将密集动作空间与分层四层级分类体系结合的GUI评判基准,支持细粒度排序评估。实验表明,无需额外标注的BBCritic-3B优于7B参数的现有最优二值模型,展现出强零样本跨平台迁移能力,验证了我们的方法论观点:GUI评判本质上是度量学习问题,而非分类问题。

原文摘要 · Abstract (English)

Test-Time Scaling (TTS), which samples multiple candidate actions and ranks them via a Critic Model, has emerged as a promising paradigm for generalist GUI agents. Its efficacy thus hinges on the critic's fine-grained ranking ability. However, existing GUI critic models uniformly adopt binary classification. Our motivational analysis of these models exposes a severe entanglement: scores for valid actions and plausible-but-invalid distractors become indistinguishable. We attribute this failure to two structural defects: Affordance Collapse--the hierarchical affordance space is compressed into 0/1 labels; and Noise Sensitivity--binary objectives overfit to noisy decision boundaries. To resolve this, we introduce BBCritic (Beyond-Binary Critic), a paradigm shift grounded in the Functional Equivalence Hypothesis. Through two-stage contrastive learning, BBCritic aligns instructions and actions in a shared Affordance Space, recovering the hierarchical structure that binary supervision flattens. We also present BBBench (Beyond-Binary Bench), the first GUI critic benchmark that pairs a dense action space with a hierarchical four-level taxonomy, enabling fine-grained ranking evaluation. Experimental results show that BBCritic-3B, trained without any extra annotation, outperforms 7B-parameter SOTA binary models. It demonstrates strong zero-shot transferability across platforms and tasks, supporting our methodological view: GUI critique is fundamentally a metric-learning problem, not a classification one.

GUI智能体度量学习非二值评判零样本迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。