为工具型AI代理的攻击危害设计七级量化评分标准,超越单纯成败判断。
Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

- 基于动作可逆性、越界程度和权限扩展设计七级危害评分体系
- 实测发现零成功率防御仍存在外部越界泄露,二进制指标无法捕捉
- 评委与人工判定高度一致,但对权限升级链条识别有系统盲区
现有代理红队基准仅以成功或失败的二元标签记录攻击结果,我们指出这种单一比特信息忽略了防御者最需了解的内容——实际行为的危害程度。为此提出一种动作分级危害量表,依据执行动作是否可逆、是否跨域影响他人、是否提升权限,在七级序数尺度(L0至L6)上对工具调用轨迹进行评分。该量表通过两种方式计算:一是基于轨迹和攻击目标的确定性规则引擎,二是由三位前沿语言模型裁判根据无标签轨迹描述独立打分。在AgentDojo工作空间套件上的四类受害模型与两组防御策略测试中,严重性评分揭示了三个二进制指标无法暴露的案例,包括一项报告攻击成功率0%却仍允许未过滤工具造成外部越界泄露的防御机制。裁判小组与规则引擎达成高阶序数一致性(Krippendorff's alpha = 0.91),但存在系统性盲点,尤其在识别权限升级链时表现不足。与已有危害分类、任务完成度测试、执行层安全评估或感知严重性的仿真不同,本工作贡献的是一个可复用、基于实际日志轨迹的严重性评估工具,所有代码、提示模板及每轮实验日志均已公开。
原文摘要 · Abstract (English)
Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that this binary attack-success rate discards the information a defender most needs, namely how harmful the resulting action was. We introduce an action-graded harm rubric that scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6) according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege. We compute the scale two ways: a deterministic oracle that reads the trajectory and the attacker's stated goal, and a panel of three frontier language-model judges that read a tag-free account of the same trajectory. Across four victim models and two defenses on the AgentDojo workspace suite, severity grading exposes three cases the binary metric hides, including a defense that reports a zero attack-success rate while still permitting an externally visible cross-scope leak through an unfiltered tool. The judge panel reproduces the oracle with high ordinal agreement (Krippendorff's alpha = 0.91) but shares systematic blind spots that we characterize, most notably a failure to recognize escalation chains. Unlike prior work that provides harm taxonomies, harmful-task completion tests, execution-level safety benchmarks, or severity-aware simulation, our contribution is a reusable, trace-grounded severity instrument applied to the actual actions recorded in existing red-team logs. All code, prompts, and per-episode logs are released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。