arXiv:2501.19180cs.CRcs.AI2025-01被引 11

用推理提前识别恶意请求,让模型主动拒绝危险指令。

Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning

  • 让模型先分析请求意图,再决定是否回应
  • 对未知攻击类型防御能力提升37%,误拒率降21%
  • 适合需要高安全性的对话系统与内容平台

大型语言模型在广泛应用中仍易受越狱攻击,导致生成不当内容。传统防御方法如拒答和对抗训练难以覆盖边缘情况或罕见领域,面对复杂攻击仍显脆弱。本文提出一种新防御策略——安全思维链(SCoT),利用大模型的推理能力,主动评估输入请求的潜在危害性,而非简单屏蔽。SCoT将任意拒答训练数据扩展为对请求意图的批判性分析,通过主动推理增强模型在未涵盖有害查询与场景下的泛化能力。同时生成详细拒答,说明违反的具体规则。对比实验表明,SCoT显著优于现有防御方案,在分布外问题和对抗操纵下漏洞减少37%,误拒率下降21%,且保持强泛化性能。

原文摘要 · Abstract (English)

Large language models (LLMs) are vital for a wide range of applications yet remain susceptible to jailbreak threats, which could lead to the generation of inappropriate responses. Conventional defenses, such as refusal and adversarial training, often fail to cover corner cases or rare domains, leaving LLMs still vulnerable to more sophisticated attacks. We propose a novel defense strategy, Safety Chain-of-Thought (SCoT), which harnesses the enhanced \textit{reasoning capabilities} of LLMs for proactive assessment of harmful inputs, rather than simply blocking them. SCoT augments any refusal training datasets to critically analyze the intent behind each request before generating answers. By employing proactive reasoning, SCoT enhances the generalization of LLMs across varied harmful queries and scenarios not covered in the safety alignment corpus. Additionally, it generates detailed refusals specifying the rules violated. Comparative evaluations show that SCoT significantly surpasses existing defenses, reducing vulnerability to out-of-distribution issues and adversarial manipulations while maintaining strong general capabilities.

越狱防御安全推理LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。