arXiv:2608.09476cs.CRcs.AI2026-08

测试协作智能体在任务中泄露数据或越权操作的风险,发现多数模型易被攻破。

ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents

论文配图:ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents
图 1 · 摘自论文原文
  • 通过可自演化任务对齐的对抗样本评估智能体行为安全性
  • 15个大模型攻击成功率最高达94.4%,显示严重安全漏洞
  • 适合研究安全评测、对抗攻击或可信AI的开发者与研究人员

协作智能体在完成良性任务时可能泄露隐私数据、操纵未授权状态或调用非法API。本文定义行为安全,并提出ActBench——一个基于执行轨迹而非最终响应的自演化基准。每个测试案例包含一个良性任务及其对抗变体,保持指令、配置、初始状态、评分模型和可信记录一致,仅注入任务可达的恶意载荷。该基准涵盖213个场景、600个案例,覆盖15种风险行为、6个执行环境和48个网页服务API。为突破静态载荷局限,提出奖励引导的束搜索方法,联合优化攻击效果与任务完成度;通过反思诊断失败节点并指导载荷修正。此外,设计双证据验证机制,结合日志证据与大模型轨迹分析双重验证安全性与有效性。我们在24,000条执行轨迹上评估了15个大模型和6个开源协作智能体。固定基模型下,攻击成功率在73.7%至94.4%之间;固定约束框架下,模型间差异为10.1%至94.4%。结果表明模型间差异大于框架间差异,且所有测试框架均面临高成功率攻击。基准已开源:https://github.com/zjuicsr/ActBench。

原文摘要 · Abstract (English)

Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API. We define behavioral safety and introduce ActBench, a self-evolving benchmark that evaluates such behavior risk from execution trajectories rather than final responses. Each case pairs a benign task with an adversarial variant that preserves its instruction, configuration, initial state, rating model, and trusted records while injecting a task-reachable payload. ActBench contains 600 cases from 213 scenarios, spanning 15 risk behaviors, six execution spaces, and 48 web-service APIs.To move beyond static payloads, we propose a reward-guided beam search method that jointly optimizes attack effectiveness and task utility, while reflection diagnoses failed execution checkpoint and guides payload revision. Besides, we propose a dual evidence verification mechanism that verifies agent execution safety and utility through log evidence and LLM-based trajectory evidence.We evaluate 15 LLMs and 6 open-source cowork agents over 24,000 trajectories. Under a fixed harness, attack success rates ranges from 10.1% to 94.4% across models, while under a fixed base model, they range from 73.7% to 94.4% across agents.These results show greater variation across models than agent harness, while attacks remain highly successful across all tested harnesses.Our benchmark is released at: https://github.com/zjuicsr/ActBench.

行为安全智能体评测对抗攻击大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。