揭示大模型越狱机制，提出新防御方法有效拦截恶意输出
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models
- 发现越狱通过将有害激活移出安全边界实现
- 新防御方法在多个测试中拦截率超98%，性能损失低于2%
- 针对低层与中层激活进行精准干预，兼顾安全性与通用能力
大型语言模型（LLMs）的越狱攻击是重大安全隐患，可能导致生成有害内容。然而，当前对越狱机制的理解仍不充分，难以制定有效防御策略。本文通过对七种不同越狱方法的全面分析，发现其本质源于观测样本不足。我们提出‘安全边界’概念，发现越狱会使有害激活偏离该边界，导致模型对有害信息敏感度下降。研究还表明，低层和中层在激活偏移中起关键作用，而深层影响较小。基于此，我们提出新型防御方法——激活边界防御（ABD），通过自适应约束激活在安全边界内，并使用贝叶斯优化仅在低层和中层施加防御。实验显示，ABD在多个基准上对各类越狱攻击的防御成功率（DSR）平均超过98%，且对模型通用能力的影响小于2%。
原文摘要 · Abstract (English)
Jailbreaking in Large Language Models (LLMs) is a major security concern as it can deceive LLMs to generate harmful text. Yet, there is still insufficient understanding of how jailbreaking works, which makes it hard to develop effective defense strategies. We aim to shed more light into this issue: we conduct a detailed large-scale analysis of seven different jailbreak methods and find that these disagreements stem from insufficient observation samples. In particular, we introduce \textit{safety boundary}, and we find that jailbreaks shift harmful activations outside that safety boundary, where LLMs are less sensitive to harmful information. We also find that the low and the middle layers are critical in such shifts, while deeper layers have less impact. Leveraging on these insights, we propose a novel defense called \textbf{Activation Boundary Defense} (ABD), which adaptively constrains the activations within the safety boundary. We further use Bayesian optimization to selectively apply the defense method to the low and the middle layers. Our experiments on several benchmarks show that ABD achieves an average DSR of over 98\% against various forms of jailbreak attacks, with less than 2\% impact on the model's general capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。