提出GUITester框架，让AI自动发现GUI缺陷，准确率超现有方法。
GUITester: Enabling GUI Agents for Exploratory Defect Discovery
- 用双模块分离导航与检测，主动探查异常
- 在143个任务上实现48.9%的缺陷发现F1分数
- 适合自动化测试、软件质量保障研究者使用
探索式GUI测试对软件质量至关重要，但人工成本高。多模态大模型代理虽能良好导航，却因两大问题无法自主发现缺陷：目标导向遮蔽（优先完成任务而非报告异常）和执行偏差归因（将系统缺陷误判为代理错误）。为此，我们首先构建了首个交互式基准GUITestBench，涵盖26个缺陷的143项任务。随后提出GUITester多代理框架，通过两个模块解耦导航与验证：(i) 计划-执行模块（PEM）主动注入测试意图探测缺陷；(ii) 分层反思模块（HRM）通过分析交互历史解决归因歧义。GUITester在GUITestBench上达到48.90% F1分数（Pass@3），优于最先进基线（33.35%）。本工作证明了自主探索式测试的可行性，并为未来GUI质量保障提供坚实基础。
原文摘要 · Abstract (English)
Exploratory GUI testing is essential for software quality but suffers from high manual costs. While Multi-modal Large Language Model (MLLM) agents excel in navigation, they fail to autonomously discover defects due to two core challenges: \textit{Goal-Oriented Masking}, where agents prioritize task completion over reporting anomalies, and \textit{Execution-Bias Attribution}, where system defects are misidentified as agent errors. To address these, we first introduce \textbf{GUITestBench}, the first interactive benchmark for this task, featuring 143 tasks across 26 defects. We then propose \textbf{GUITester}, a multi-agent framework that decouples navigation from verification via two modules: (i) a \textit{Planning-Execution Module (PEM)} that proactively probes for defects via embedded testing intents, and (ii) a \textit{Hierarchical Reflection Module (HRM)} that resolves attribution ambiguity through interaction history analysis. GUITester achieves an F1-score of 48.90\% (Pass@3) on GUITestBench, outperforming state-of-the-art baselines (33.35\%). Our work demonstrates the feasibility of autonomous exploratory testing and provides a robust foundation for future GUI quality assurance~\footnote{Our code is now available in~\href{https://github.com/ADaM-BJTU/GUITestBench}{https://github.com/ADaM-BJTU/GUITestBench}}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。