不同防御机制对大模型安全威胁的防护效果各异，且存在被改写绕过的风险。
Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing
- 通过构建四种防御配置的模拟环境，量化各防御手段对十大威胁的覆盖能力。
- 仅拒绝机制可消除越狱和系统提示泄露，预算控制能阻止敏感信息暴露与无限制使用。
- 改写攻击使拒绝机制防护率下降15-25个百分点，预算控制则保持稳定。
生产级大模型应用通常叠加多种防御机制——拒绝词过滤、令牌预算控制、模型白名单、速率限制、工具注册认证等——但现有攻防测试基准仅报告单一覆盖率，掩盖了各类防御对具体威胁的实际作用。本文开展归因分析：在21个基线扫描代理基础上新增4个具备OWASP-LLM-Top-10认知的代理，针对四个合成的大模型端点进行测试：$L_0$（无防御）、$L_1$（仅拒绝）、$L_2$（仅预算）、$L_3$（完整防御栈）。其中 $L_1$ 与 $L_2$ 为单轴消融，互不包含；$L_3$ 为其并集外加工具注册认证与凭证清洗。在 $N=10$ 次重复实验中，每项威胁的发现数清晰可辨：仅拒绝机制可消除全部 LLM01（越狱）与 LLM07（系统提示泄露）；仅预算控制可消除全部 LLM02（敏感信息泄露）与 LLM10（无边界消耗），通过终止多步序列实现；而 LLM06（过度自主性）需完整防御栈。进一步测试改写攻击的脆弱性：使用300条Gemini生成的改写文本（$K=5$，基于60模板脆弱性语料库），$L_1$ 的拒绝阻断率在 LLM01 上下降15个百分点，在 LLM07 上下降25个百分点。引入第五个目标 $L_4$-real，将后端替换为Gemini-2.5-flash，其余配置与 $L_3$ 一致，其表现与 $L_1$ 完全匹配，表明除正则表达式外，模型对齐未带来可测量贡献（非普遍结论）。预算控制则无下降（扣除速率限制下限后为0个百分点）。静态基准的拒绝白名单可被大模型驱动的改写器击败而不改变攻击意图；而预算控制能抵御此类变异。
原文摘要 · Abstract (English)
Production LLM applications stack several defense families -- refusal-phrase filters, token-budget controls, model allowlists, rate limits, tool-registry authentication -- yet existing breach-and-attack-simulation (BAS) benchmarks report a single aggregate coverage number, hiding which family closes which threat. We measure attribution. We add four OWASP-LLM-Top-10-aware agents to a 21-agent baseline scanner and target a lattice of four synthetic LLM endpoints: $L_0$ (no defenses), $L_1$ (refusal-only), $L_2$ (budget-only), and $L_3$ (full stack). $L_1$ and $L_2$ are sibling single-axis ablations, not subsets of each other; $L_3$ is their union plus tool-registry authentication and credential scrubbing. Across $N=10$ replications, the per-OWASP finding count is clean: refusal alone removes all LLM01 (jailbreak) and LLM07 (system-prompt leakage) findings; budget alone removes all LLM02 (sensitive-info disclosure) and LLM10 (unbounded consumption) findings by terminating multi-step sequences; LLM06 (excessive agency) requires the full stack. We probe brittleness under paraphrasing: with 300 Gemini-generated paraphrases ($K=5$ over a 60-template brittleness corpus), $L_1$ refusal block rate falls 15 pp on LLM01 and 25 pp on LLM07. A fifth target, $L_4$-real, swaps the stub backend for Gemini-2.5-flash behind the same $L_3$ regex and matches $L_1$ exactly, indicating no measurable alignment contribution beyond the regex (not a general claim about alignment). Budget controls show no drop (0 pp once the rate-limit floor is factored out). A refusal whitelist that clears a static benchmark can be defeated by an LLM-driven paraphraser without changing attack intent; a budget control resists the same mutation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。