首个评估网页智能体恶意行为的基准，揭示大模型易被滥用风险。
SafeArena: Evaluating the Safety of Autonomous Web Agents
- 构建250个有害任务，涵盖五类真实危害场景。
- GPT-4o和Qwen-2完成超27%恶意请求，合规率极低。
- 提出风险评估框架，适合安全研究者与开发者参考。
基于大语言模型的智能体在完成网络任务方面日益熟练，但随之而来的潜在滥用风险也日益突出，例如在在线论坛发布虚假信息或在网站上销售违禁品。为评估此类风险，我们提出了SafeArena——首个专注于网页智能体故意滥用行为的基准。该基准包含四个网站上的250个安全任务与250个有害任务，有害任务被划分为五大类别：虚假信息、非法活动、骚扰、网络犯罪和社会偏见，以评估现实中的滥用可能。我们对GPT-4o、Claude-3.5 Sonnet、Qwen-2-VL 72B和Llama-3.2 90B等主流网页智能体进行了测试。为系统评估其对有害任务的敏感性，我们引入了代理风险评估框架，将行为分为四个风险等级。结果显示，智能体对恶意请求表现出惊人配合度，其中GPT-4o和Qwen-2分别完成34.7%和27.3%的有害请求。研究凸显了对网页智能体进行安全对齐的紧迫需求。基准数据集可访问：https://safearena.github.io
原文摘要 · Abstract (English)
LLM-based agents are becoming increasingly proficient at solving web-based tasks. With this capability comes a greater risk of misuse for malicious purposes, such as posting misinformation in an online forum or selling illicit substances on a website. To evaluate these risks, we propose SafeArena, the first benchmark to focus on the deliberate misuse of web agents. SafeArena comprises 250 safe and 250 harmful tasks across four websites. We classify the harmful tasks into five harm categories -- misinformation, illegal activity, harassment, cybercrime, and social bias, designed to assess realistic misuses of web agents. We evaluate leading LLM-based web agents, including GPT-4o, Claude-3.5 Sonnet, Qwen-2-VL 72B, and Llama-3.2 90B, on our benchmark. To systematically assess their susceptibility to harmful tasks, we introduce the Agent Risk Assessment framework that categorizes agent behavior across four risk levels. We find agents are surprisingly compliant with malicious requests, with GPT-4o and Qwen-2 completing 34.7% and 27.3% of harmful requests, respectively. Our findings highlight the urgent need for safety alignment procedures for web agents. Our benchmark is available here: https://safearena.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。