检验本地大模型防御机制在语义攻击下的失效原因
Breaking the Assumptions: Auditing Input-Side Jailbreak Defenses Against Semantic Attacks
- 逐项分析防御依赖的假设条件并验证其有效性
- 在6个开源模型上测试13,800次，发现多数防御因假设被突破而失效
- 揭示防御漏洞根源，适合研究模型安全与对抗攻击者参考
本地部署的大语言模型（如通过Ollama）缺乏API服务模型的审核与滥用检测机制，因此其安全性依赖于各类防御措施，而这些措施的有效性取决于其设计所基于的假设。本文对本地部署模型的防御机制在越狱攻击下的表现进行了系统性审计。部分防御提供形式化保证（如SmoothLLM、Erase-and-Check、Sequential Monitors），另一些则依赖经验检测（如Semantic Smoothing、Self-Denoised Smoothing、Perplexity Filtering）。我们不仅指出防御失败，更追溯每项失败背后的假设：针对每个防御提取其依赖的前提条件，推导出该条件被违反时应产生的实证模式，并在6个开源模型（参数规模14B至35B）上，使用来自40多个公开来源的100个越狱提示，共生成13,800条评估记录进行验证。
原文摘要 · Abstract (English)
Locally deployed Large Language Models (LLMs) via inference engines such as Ollama run without the moderation and abuse detection present in API-served models. Therefore, the safety of LLMs depends on the defense mechanisms used, and their effectiveness depends on the assumptions on which they were designed. This paper does an audit of defense mechanisms under jailbreak attacks on locally deployed models. Some defenses provide formal guarantees (SmoothLLM, Erase-and-Check, Sequential Monitors), while others rely on empirical detection results (Semantic Smoothing, Self-Denoised Smoothing, Perplexity Filtering). Instead of merely observing that defenses fail, we trace each failure back to the specific assumption: for every defense, we extract the condition it relies on, derive the empirical pattern a violation should produce, and test that prediction on six open-weight models (14B to 35B parameters) with a corpus of 100 jailbreak prompts taken from more than 40 public sources, totalling 13,800 evaluation records.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。