研究大模型防御的副作用，发现不同策略各有优劣。
When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
- 按防御策略分类，分析安全与性能的权衡关系
- 规则类防御最保性能，自省式防御易误拒正常请求
- 多轮验证防御耗时最长，适合对效率要求不高的场景
jailbreak 防御对保护大语言模型至关重要，但可能带来次生代价，削弱模型实用性。本文系统研究了防御在性能影响、对良性输入的过度拒绝以及推理成本三个维度上的权衡。不将防御视为单一类别，而是按操作策略分类，考察不同策略与副作用特征的关联。在主流防御方法、常用基准数据集和代表性开源 LLM 上，发现防御极少提升下游能力，反而在安全收益与可用性、效率之间存在差异化的权衡。具体而言，规则类防御最能保持任务性能，高度保守的自省式防御常导致过度拒绝，多轮验证类防御产生最大运行时开销。研究结果为评估防御副作用提供了基准，并为部署时选择合适防御提供实用指导。
原文摘要 · Abstract (English)
Jailbreak defenses are essential for protecting large language models (LLMs), but they can also introduce secondary costs that weaken model utility. We present a systematic study of these defense trade-offs along three dimensions: performance impact, over-refusal on benign inputs, and inference cost. Rather than treating defenses as a single class, we organize them by operational strategy and examine how different strategies correlate with different side-effect profiles. Across state-of-the-art defense methods, widely used benchmark datasets, and representative open-source LLMs, we find that defenses rarely improve downstream capability, but instead vary in how they trade safety gains against usability and efficiency. In particular, rule-based defenses best preserve task performance, highly conservative self-reflective defenses often increase over-refusal, and multi-round defenses incur the largest runtime overhead. These results provide both a benchmark for evaluating defense side effects and practical guidance for selecting defenses under deployment constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。