提出新基准EvoGUI，专测GUI状态转换理解能力。
EvoGUI: An Evolution-Aware Benchmark for GUI State-Transition Understanding
- 用轨迹生成三类问答任务，无须额外标注
- 28模型零样本测试最高仅60.4分，性能提升空间大
- 适合评估视觉语言模型对界面变化的推理能力
GUI智能体需理解操作如何改变界面状态，但端到端成功率将状态推理与感知、定位、规划和恢复能力混杂。本文提出EvoGUI，一种诊断性框架，将归一化的GUI轨迹转化为三种互补的视觉问答探针：时间顺序判断、逆向动作/值预测、对比单步后继识别。标签基于轨迹顺序和日志操作生成，无需额外任务标注。基于Mind2Web和WebLINX构建EvoGUI-Bench，涵盖120个领域共3000个实例，对28种视觉-语言模型配置进行零样本评估。最强模型仅达60.4 EvoGain，模型规模与GUI专业化均无法可靠预测性能。结果表明EvoGUI-Bench可作为端到端评估的可扩展诊断补充，同时揭示状态转换理解仍存在巨大提升空间。源代码已公开于https://github.com/Yyhhh6/EvoGUI。
原文摘要 · Abstract (English)
GUI agents must reason about how actions transform interface states, but end-to-end success rates entangle this ability with perception, grounding, planning, and recovery. We introduce EvoGUI, a diagnostic framework that converts normalized GUI trajectories into three complementary visual question answering probes: temporal ordering, inverse action/value prediction, and contrastive one-step successor discrimination. Their labels are derived from trajectory order and logged actions, requiring no additional task-label annotation after trajectory normalization. We instantiate EvoGUI-Bench from Mind2Web and WebLINX, yielding 3,000 instances across 120 domains, and evaluate 28 vision-language model configurations zero-shot. The strongest model reaches only 60.4 EvoGain, while model scale and GUI specialization do not reliably predict performance. These results establish EvoGUI-Bench as a scalable diagnostic complement to end-to-end GUI-agent evaluation while exposing substantial headroom in state-transition understanding. The source code is publicly available at https://github.com/Yyhhh6/EvoGUI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。