针对小模型越狱攻击,提出基于激活值的轻量级防御方法。
GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Models
- 通过分析不同层激活模式,识别越狱行为特征。
- 在7个小型模型上测试,有效拦截9种越狱攻击。
- 轻量级设计适合边缘设备部署,兼顾安全与效率。
小型语言模型(SLMs)因其低计算成本和低延迟,成为替代大型语言模型(LLMs)的高效选择,适用于资源受限的边缘设备。然而,现有越狱攻击防御措施对多样化攻击鲁棒性不足,主要源于对模型各层内部表征理解不充分。本文对7个SLMs和3个LLMs上的9种越狱攻击进行了全面实证研究,发现SLMs仍极易受恶意提示绕过安全对齐。通过分析不同层的隐藏状态激活,揭示了不同输入类型在内部表示空间中形成可区分模式。基于此,提出GUARD-SLM——一种轻量级的令牌激活过滤方法,在推理阶段直接作用于表示空间,实现恶意提示的精准识别与过滤,同时保留正常请求。研究揭示了模型各层的鲁棒性差异,为小型语言模型的安全部署提供了切实可行的方向。
原文摘要 · Abstract (English)
Small Language Models (SLMs) are emerging as efficient and economically viable alternatives to Large Language Models (LLMs), offering competitive performance with significantly lower computational costs and latency. These advantages make SLMs suitable for resource-constrained and efficient deployment on edge devices. However, existing jailbreak defenses show limited robustness against heterogeneous attacks, largely due to an incomplete understanding of the internal representations across different layers of language models that facilitate jailbreak behaviors. In this paper, we conduct a comprehensive empirical study on 9 jailbreak attacks across 7 SLMs and 3 LLMs. Our analysis shows that SLMs remain highly vulnerable to malicious prompts that bypass safety alignment. We analyze hidden-layer activations across different layers and model architectures, revealing that different input types form distinguishable patterns in the internal representation space. Based on this observation, we propose GUARD-SLM, a lightweight token activation-based method that operates in the representation space to filter malicious prompts during inference while preserving benign ones. Our findings highlight robustness limitations across layers of language models and provide a practical direction for secure small language model deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。