通过压缩提示词识别恶意意图,高效防御大模型越狱攻击。
SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
- 用安全感知的提示压缩器识别输入的真实意图
- 在不改变原输入的前提下提升安全防护效果
- 计算开销极小,适合实际部署场景
大型语言模型(LLMs)在众多应用中广泛使用,但即使经过安全对齐仍易受恶意攻击。攻击者常通过对抗性越狱提示包裹恶意指令以绕过安全防护。现有方法如对抗训练和提示重述虽能缓解问题,但常降低模型实用性或带来显著计算开销与延迟。本文提出SecurityLingua,一种基于安全导向提示压缩的高效防御机制。该方法训练一个提示压缩器,专注于识别输入提示中的真实意图,尤其是恶意意图。压缩后的意图作为系统提示传递给目标LLM,帮助其识别请求本质并激活内置安全机制。该方案保持原始输入不变,确保用户体验一致,同时仅引入可忽略的额外计算与令牌开销。实验表明,SecurityLingua能有效防御恶意攻击,且几乎不影响模型实用性。代码已开源:https://aka.ms/SecurityLingua。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved widespread adoption across numerous applications. However, many LLMs are vulnerable to malicious attacks even after safety alignment. These attacks typically bypass LLMs' safety guardrails by wrapping the original malicious instructions inside adversarial jailbreaks prompts. Previous research has proposed methods such as adversarial training and prompt rephrasing to mitigate these safety vulnerabilities, but these methods often reduce the utility of LLMs or lead to significant computational overhead and online latency. In this paper, we propose SecurityLingua, an effective and efficient approach to defend LLMs against jailbreak attacks via security-oriented prompt compression. Specifically, we train a prompt compressor designed to discern the "true intention" of the input prompt, with a particular focus on detecting the malicious intentions of adversarial prompts. Then, in addition to the original prompt, the intention is passed via the system prompt to the target LLM to help it identify the true intention of the request. SecurityLingua ensures a consistent user experience by leaving the original input prompt intact while revealing the user's potentially malicious intention and stimulating the built-in safety guardrails of the LLM. Moreover, thanks to prompt compression, SecurityLingua incurs only a negligible overhead and extra token cost compared to all existing defense methods, making it an especially practical solution for LLM defense. Experimental results demonstrate that SecurityLingua can effectively defend against malicious attacks and maintain utility of the LLM with negligible compute and latency overhead. Our code is available at https://aka.ms/SecurityLingua.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。