arXiv:2412.10423cs.CLcs.AI2024-12被引 2

用规则提示提前识别有害内容,让大模型更安全地回应。

Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

  • 先分析提问风险,生成安全建议再让主模型回应
  • 平均降低34.17%攻击成功率,且不影响正常问答
  • 无需重训主模型,适配多种大语言模型

尽管大语言模型(LLMs)具备对齐机制,但仍面临不断演化的越狱攻击,可能破坏其安全性。现有方法在训练效率和泛化能力方面存在局限(如基于人类反馈的强化学习与红队测试)。为此,我们提出新型防御范式GuidelineLLM,它在主模型响应前先识别查询中的潜在风险,将风险归纳为指导性建议,并传递给主模型。该方法无需对主模型进行额外安全微调,仅需微调GuidelineLLM,提升了跨模型适用性。实验表明,GuidelineLLM可显著降低攻击成功率(平均降低34.17%),同时保持主模型处理良性查询的能力。代码已开源。

原文摘要 · Abstract (English)

Despite being empowered with alignment mechanisms, large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks that can compromise their alignment mechanisms. This vulnerability poses significant risks to real-world applications. Existing work faces challenges in both training efficiency and generalization capabilities (i.e., Reinforcement Learning from Human Feedback and Red-Teaming). Developing effective strategies to enable LLMs to resist continuously evolving jailbreak attempts represents a significant challenge. To address this challenge, we propose a novel defensive paradigm called GuidelineLLM, which assists LLMs in recognizing queries that may have harmful content. Before LLMs respond to a query, GuidelineLLM first identifies potential risks associated with the query, summarizes these risks into guideline suggestions, and then feeds these guidelines to the responding LLMs. Importantly, our approach eliminates the necessity for additional safety fine-tuning of the LLMs themselves; only the GuidelineLLM requires fine-tuning. This characteristic enhances the general applicability of GuidelineLLM across various LLMs. Experimental results demonstrate that GuidelineLLM can significantly reduce the attack success rate (ASR) against LLM (an average reduction of 34.17\% ASR) while maintaining the usefulness of LLM in handling benign queries. The code is available at https://github.com/sqzhang-lazy/GuidelineLLM.

安全防御越狱攻击提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。