通过分析模型隐藏状态,提前识别并拦截越狱攻击,无需额外推理开销。
HSF: Defending against Jailbreak Attacks with Hidden State Filtering
- 利用大模型隐藏状态的聚类差异,将防御转为前置分类任务。
- 在三款模型上对六种前沿越狱攻击的防御成功率显著提升。
- 几乎不增加计算开销,且不影响正常用户请求的响应质量。
随着大语言模型在聊天机器人和内容生成等日常应用中的广泛部署,确保输出符合人类价值观并避免有害内容的努力日益加强。然而,日益复杂的越狱攻击正威胁这一对齐性,旨在诱导不安全输出。当前防御方法或聚焦于提示重写或检测,受限于越狱提示的多样设计;或依赖输出控制与检测,计算成本高昂,需进行大模型推理。因此,设计一种可在推理前抵御多种越狱提示的防御方法至关重要。我们观察到,越狱攻击、安全查询与有害查询在大模型的隐藏状态表示空间中呈现不同的聚类模式。这表明可借助隐藏状态表征能力,分析模型潜在行为并主动干预防御。本文提出基于隐藏状态过滤器(HSF)的越狱攻击防御策略,这是一种无损的架构级防御机制,可在推理前预判并拒绝恶意输入。通过附加插件模块激活其防御能力,将防御任务转化为分类问题。在两个基准数据集上,使用三种不同大模型的实验结果表明,HSF显著提升了对六种前沿越狱攻击的鲁棒性,大幅降低攻击成功率,同时对良性用户查询影响极小,且推理开销可忽略不计,优于现有防御基线。代码与数据已公开于 https://anonymous.4open.science/r/Hidden-State-Filtering-8652/
原文摘要 · Abstract (English)
With the growing deployment of LLMs in daily applications like chatbots and content generation, efforts to ensure outputs align with human values and avoid harmful content have intensified. However, increasingly sophisticated jailbreak attacks threaten this alignment, aiming to induce unsafe outputs. Current defense efforts either focus on prompt rewriting or detection, which are limited in effectiveness due to the various design of jailbreak prompts, or on output control and detection, which are computationally expensive as they require LLM inference. Therefore, designing a pre-inference defense method that resists diverse jailbreak prompts is crucial for preventing LLM jailbreak attacks. We observe that jailbreak attacks, safe queries, and harmful queries exhibit different clustering patterns within the LLM's hidden state representation space. This suggests that by leveraging the LLM's hidden state representational capabilities, we can analyze the LLM's forthcoming behavior and proactively intervene for defense. In this paper, we propose a jailbreak attack defense strategy based on a Hidden State Filter (HSF), a lossless architectural defense mechanism that enables the model to preemptively identify and reject adversarial inputs before the inference process begins. We activate its defensive potential through an additional plugin module, effectively framing the defense task as a classification problem. Experimental results on two benchmark datasets, utilizing three different LLMs, show that HSF significantly enhances resilience against six cutting-edge jailbreak attacks. It significantly reduces the success rate of jailbreak attacks while minimally impacting responses to benign user queries, with negligible inference overhead, and outperforming defense baselines.Our code and data are available at https://anonymous.4open.science/r/Hidden-State-Filtering-8652/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。