为网页智能体打造可泛化的安全防护系统,解决其误操作风险。
WebGuard: Building a Generalizable Guardrail for Web Agents
- 构建首个面向网页智能体的风险评估数据集,标注4939条动作
- 现有大模型预测准确率不足60%,高风险动作漏检率超80%
- 微调后模型准确率提升至80%,仍不满足高安全场景需求
由大语言模型驱动的自主网页智能体虽大幅提升效率,却面临执行意外或有害操作的前沿风险。为此,本文提出WebGuard——首个支持网页智能体行为风险评估与防护机制开发的综合性数据集。该数据集聚焦状态变更类操作的后果预测,涵盖22个不同领域、193个网站的4939条人工标注动作,采用全新的三级风险分类体系(SAFE、LOW、HIGH)。数据集提供明确的训练与测试划分,支持多种泛化场景下的评估。初步评估显示,即使领先的大模型在预测动作结果时准确率也低于60%,对高风险动作的召回率不足60%。为此,我们基于WebGuard对专用防护模型进行微调。在多个泛化设置下评估发现,经微调的Qwen2.5VL-7B模型性能显著提升:准确率从37%升至80%,高风险动作召回率从20%增至76%。尽管如此,其表现仍难以满足高安全场景中近乎完美的可靠性要求。
原文摘要 · Abstract (English)
The rapid development of autonomous web agents powered by Large Language Models (LLMs), while greatly elevating efficiency, exposes the frontier risk of taking unintended or harmful actions. This situation underscores an urgent need for effective safety measures, akin to access controls for human users. To address this critical challenge, we introduce WebGuard, the first comprehensive dataset designed to support the assessment of web agent action risks and facilitate the development of guardrails for real-world online environments. In doing so, WebGuard specifically focuses on predicting the outcome of state-changing actions and contains 4,939 human-annotated actions from 193 websites across 22 diverse domains, including often-overlooked long-tail websites. These actions are categorized using a novel three-tier risk schema: SAFE, LOW, and HIGH. The dataset includes designated training and test splits to support evaluation under diverse generalization settings. Our initial evaluations reveal a concerning deficiency: even frontier LLMs achieve less than 60% accuracy in predicting action outcomes and less than 60% recall in lagging HIGH-risk actions, highlighting the risks of deploying current-generation agents without dedicated safeguards. We therefore investigate fine-tuning specialized guardrail models using WebGuard. We conduct comprehensive evaluations across multiple generalization settings and find that a fine-tuned Qwen2.5VL-7B model yields a substantial improvement in performance, boosting accuracy from 37% to 80% and HIGH-risk action recall from 20% to 76%. Despite these improvements, the performance still falls short of the reliability required for high-stakes deployment, where guardrails must approach near-perfect accuracy and recall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。