构建真实网页引导生成基准,自动识别操作目标并生成精准提示。
GuideWeb: A Benchmark for Automatic In-App Guide Generation on Real-World Web UIs
- 基于网页元素定位与用户意图理解,自动生成页面级引导
- 目标元素预测准确率达30.79%,意图生成BLEU达44.94,引导文本生成21.34
- 适合UI自动化、数字助手与低代码平台研究者参考
数字采用平台(DAP)通过网页叠加层提供操作指引和上下文提示,帮助用户使用复杂网站。尽管现代DAP工具使非专家也能创建指引,但维护成本高,因网页布局和功能持续变化,需反复手动更新与重新标注。本文提出「GuideWeb」,一个面向真实世界网页UI的自动应用内引导生成基准。该任务被定义为:根据网页内容选择引导目标元素,并生成与用户意图对齐的简洁引导文本。我们还设计了综合性评估体系,联合衡量目标元素选择准确率与生成意图及引导文本的质量。实验表明,所提出的GuideWeb Agent在目标元素预测上达到30.79%准确率,意图生成的BLEU得分为44.94,引导文本生成得分为21.34。现有基线表现显著更差,凸显自动引导生成仍具挑战性,未来还需进一步突破方可实际部署。
原文摘要 · Abstract (English)
Digital Adoption Platform (DAP) provide web-based overlays that deliver operation guidance and contextual hints to help users navigate complex websites. Although modern DAP tools enable non-experts to author such guidance, maintaining these guides remains labor-intensive because website layouts and functionalities evolve continuously, which requires repeated manual updates and re-annotation. In this work, we introduce \textbf{GuideWeb}, a new benchmark for automatic in-app guide generation on real-world web UIs. GuideWeb formulates the task as producing page-level guidance by selecting \textbf{guide target elements} grounded in the webpage and generating concise guide text aligned with user intent. We also propose a comprehensive evaluation suite that jointly measures the accuracy of guide target element selection and the quality of generated intents and guide texts. Experiments show that our proposed \textbf{GuideWeb Agent} achieves \textbf{30.79\%} accuracy in guide target element prediction, while obtaining BLEU scores of \textbf{44.94} for intent generation and \textbf{21.34} for guide-text generation. Existing baselines perform substantially worse, which highlights that automatic guide generation remains challenging and that further advances are necessary before such systems can be reliably deployed in real-world settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。